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PREFACE 


1 he prc-sent work is the fourth in the series of volumes which 
constitute the refx>rt of the C>>mmission on the Social Studies in 
the Schot)ls. The first three volumes present successively a view of 
the problem, its historical background, and the concern of the pub- 
lic in the problem. Thi.s, the fourth, presents the experience of the 
(>)mmis,sion in its study of objective tests and measurements in the 
field of the .social studies. 

The work of test construction w'as at first directly under the charge 
of the Commission as a whole. There was at that time sfjme hope 
that the whole field of social science instruction might be submitted 
to objective measurement with important implications for each divi- 
sion of the work. I hcrc was also some awareness of the long time 
required for the preliminary analytical work. For these reasons the 
('ommission provided early for this work on tests. Early reports of 
prr)grcss, however, indicated that the work of test construction 
involved a high degree of technical knowledge. The Commission 
accordingly apjxtintcd an Advisory Committee consisting of Frank 
W. Rallou, Chairman, Isaiah Bowman, Howard C. Hill, Ernest 
Horn. Henry Johnson, A. C. Krcy. and Ben D. Wood to have 
immediate direction of test con.struction. Truman L. Kelley, psy- 
chologist, advi.scd the Committee. This Committee considered the 
tests constructed by Edgar B. Wesley, Mary G. Kelty, Marion Clark, 
and tiiosc of Edith P. Parker and R D Calkins, Its members were 
also consulted for advice on the portions of this volume contributed 
by T. L. Kelley and A. ('. Krey. Obviously, therefore, the Commit- 
tee has only a partial responsibility for the material here presented. 
The authors of s{>ccific sections as given in the Table of Contents 
arc rcsf^xjnsible for .such .sections and the Chairman of the Commis- 
sion is responsible for any matter not otherwise credited. 

It will be difficult, if not impossible, to acknowledge all the help 
received in the preparation of this volume. Numerous tests were 
analyzed; .some already siandardiz.ed, others in earlier stages of 
development, some published, others in less {permanent form, each 
the product of more or less cooperation. The analytical work of find- 
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ing items for use in tests, the independent checking of these items, 
the judgment of the items by consensus of opinion, the prelimi- 
nary construction of test questions, critical judgment of such ques- 
tions, the trial of these questions under a sufficiently varied set of 
school conditions, as well as the evaluation of statistical findings, all 
required the assistance of a number of scholars and teachers. This 
assistance has been rendered at intervals over a period of nearly five 
years. Under the circumstances, the chance that some even of those 
who aided directly have been overlooked is too great to justify the 
hope that all assistance has been acknowledged. We can only hope 
that such omissions will be discovered before the final volume of 
the Commission’s report is published so that due acknowledgment 
may be made at that time. 

The late Frances E. Baldwin, O. M. Dickerson, Mary Gold, Don- 
ald L. McMurry, Joseph R. Strayer, Edith fi. Ware, and Edgar B. 
Wesley were attached to the staffi of the Investigation for varying 
periods and devoted much of their attention to this work. Dorothy 
Bovee, Gladys Boyington, Dorothy Houston, Herman Muelder, 
Renata Pccinovsky, Lester N. Recktenwald, William Shepherd, 
Garland Taylor, and Olga Wold, teachers of varied experience who 
were continuing their graduate study, have aided in this work for 
definite periods of time. 

We arc indebted for various forms of voluntary assistance to the 
following of our colleagues throughout the country: H. S, Anderson, 
John Ascltine, Elizabeth Baker, R. Bell, Robert C. Binkley, H, H. 
Bixler, Ernest L. Bogart, Witt Bowden, D. S. Brainard, Harold H. 
Burbank, Kathryn E. C. Carrigan, J. H. Carman, Thomas N. Carver, 
P. Eugene Davenport, Edgar Dawson, Virgil E. Dickson, Evalyn 
Dixon, Theodore Estabrook, F. R. Fairchild, Halley Farmer, Fred- 
eric B. Garver, John M. Gaus, Alice N. Gibbons, Charles E. Greene, 
J. John Halverson, P. M. Hamer, J. R. Harper, W. H. Hathaway, 
Mark M. Heald, E. S. Henry, J. D. Hicks, Grace Hotchkiss, R. O. 
Hughes, Horace Kidgcr, Von E. Knisclcy, Jessie La Salle, Benjamin 
E, Lippincott, John Lund, Paul E. Lutz, I. N. Madsen, Carl A. Mel- 
by, Floyd B. Moc, DeWitt S. Morgan, Eugene A. Nifcneckcr, A. H. 

[viii] 
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Noyes, C. W. Odell, Lawrence Packard, Clarence Perkins, J. E. 
Pomfret, Arthur Repp, Edgar E. Robinson, Theodore Saam, Allan 
Saunders, Caroline Shanks, J. L. Sharfman, Richard H. Shryock, 
Russell A. Stevenson, Edna H. Stone, E. J. Sweeney, Lida Lee Tall, 
V. T. Thayer, Brian E. Tomlinson, Charles F. Towne, Alice Felt 
Tyler, Laura F. UUrick, R. S. Vaile, E. M. Violette, Carleton Wash- 
burne, W. P. Webb, Raymond Werner, Benjamin W. Wheeler, A. C. 
Wilgus, Malcolm Willey, Fremont P. Wirth, Dean A. Worcester, 
and John Wuorinen. 

Many school systems have aided in the trial stages of the several 
tests. As the tests were of an experimental nature and bore no direct 
relationship to courses as conducted in any of the schools, they could 
not be used for general examination purposes. The trial of the tests, 
therefore, was confined to limited groups of pupils in any one school 
system and the whole picture was completed by piecing together the 
returns from scattered groups. For their assistance under these cir- 
cumstances the Commission feels esi>ccially grateful to the school 
authorities of Americus, Kansas; Anoka, Minnesota; Asheville, 
North Carolina; Atlanta, Georgia; Atlantic City, New Jersey; 
Avondale, North Carolina; Benson, North Carolina; Beverley, 
Massachusetts; Bilimore, North Carolina; Bonlee, North Carolina; 
Cambridge, Massachu.setts; Carolcen, North Carolina; Chadbournc, 
North Carolina; Clitfside, North Carolina; Clinton, North Car- 
olina; Columbus, Ohio; Charleston, Massachusetts; Charleston, 
West Virginia; Hyde Park High School, Chicago, Illinois; Del- 
gado, North Carolina; Denver, Colorado; Dearborn, Michigan; 
Elgin, Illinois; Emporia, Kansas; Excelsior, Minnesota; Falcon, 
North Carolina; Faribault, Minnesota; Glassboro, New Jersey; 
Goldston, North Carolina; Grosse Point, Michigan; Haddon 
Heights, New Jersey; Henderson, North Carolina; Hendersonville, 
North Carolina; Highland Park, Michigan; Horace Mann School, 
New York City; Houston, Texas; Indianapolis, Indiana; Kansas 
City, Missouri; Kenosha, Wisconsin; Lattimorc, North Carolina; 
Lawrence School, Hewlett, New York; Long Beach, California; 
Lyons, New York; Maiden, North Carolina; Milwaukee, Wisconsin; 
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Minneapolis, Minnesota; Monroe, North Carolina; New Orleans, 
Louisiana; Newton, North Carolina; Newtonvillc, Massachusetts; 
New York City; Norwalk, Connecticut; Oakland, California; 
Old Ford, North Carolina; Overbrook, Pennsylvania; Owatonna, 
Minnesota; Pantego, North Carolina; Pen Yan, New York; 
Philadelphia, Pennsylvania; Pittshoro, North Carolina; Pittsburgh, 
Pennsylvania; Providence, Rhode Island; Reading, Massachu- 
setts; Roanoke. Virginia; Rochester, New York; Rock Springs, 
Georgia; Roseboro, North Carolina; Rutherfordton, Nr)rth Caro- 
lina; Sacramento, California; St, Louis, Missouri; St. Paul, Minne- 
sota; San Diego, California; Sanford. North Carolina; Seattle, Wash- 
ington; Shakopee, Minnesota; South Hamilton, Massachusetts; Sum- 
mit, New Jersey; Turkey, North Carolina; Urbana, Illinois; Wash- 
ington, D. C.; Washington, North Carolina; Wenham, Massa- 
chusetts; W’hiteville, North Carolina; Wilmette, Illinois; Wilming- 
ton, North Carolina; Winnetka, Illinois; Wrightsville, Georgia. 

We are also indebted for assistance on one or more of the tests to 
the following colleges: Duke University, Durham, North Carolina; 
State Teachers College, Lmjxaria, Kansas; University of California, 
Berkeley; State Teachers College, Flagstaff, Arizona; Teachers Col- 
lege, Columbia University, New York City; Harvard University, 
Cambridge, Massachusetts; Hunter College, New York City; Uni- 
versity of Louisiana, Baton Rouge; State College for Women, Monte- 
vallo, Alabama; State Normal School, Lewiston, Montana; Univer- 
sity of Illinois, Urbana; University of Minnesota, Minneapolis; Uni- 
versity of Nebraska, Lincoln; New York University, New York City; 
University of North Dakota, Grand Forks; Ohio State University, 
Columbus; Princeton University, Princeton, New Jersey; St. Olaf 
College, Northfield, Minnesota; State Teachers College, St. Cloud, 
Minnesota; Stanford University, Palo Alto, California; University of 
Tennessee, Knoxville; State Normal School, Towson, Maryland; 
State Teachers College, Virginia, Minnesota; George Peabody Col- 
lege for Teachers, Nashville, Tennessee. 

Grateful acknowledgment for criticism and help on the technical 
aspects of the problem is also made to the following: H, R. Andcr- 



PREFACE 


son, Philip A. Boyer, Prudence Cutright, Cecile White Flemming, 
Mary Harden, William C. Knoclk, E. F. Lindquist, Caroline Pfaff, 
Irene M. Steele, O. M. Stephenson, Lena C. Van Bibber, Howard 
E. Wilson, and Clifford Woody. 

It would be ingratitude indeed to leave the impression even by 
implication that the many individuals and institutions which so 
kindly gave assistance were in any way responsible for any short- 
comings which may appear in this work. Their help was given at 
various stages and on various parts of the work herein described. 
Few of them had any share in planning the work and few^er still in 
passing upon the conclusions reached. It is even possible that some 
of them would find themselves at variance with a number of the 
conclusions herein set forth. For these reasons it seems desirable to 
absolve them from any such responsibility, while, at the same time, 
acknowledging with gratitude the assistance w'hich they have ren- 
dered. 

Each of the contributions in this volume bears the name of the 
individual immediately responsible for its statements. Any evalua- 
tion of these contributions must, of necessity, be made in the light of 
the background from which the individual approached his task. 
Several of the contributions are made by scholars in educational meas- 
urement and psychology who have had very little occasion to pro- 
ceed far, or to remain in close touch, with advanced work in social 
science subjects. Other contributions are made by teachers at various 
levels of school instruction — the elementary .school, the high school, 
and college. These teachers have had various degrees of experi- 
ence with the technical as^xxts of the new type test. Statistical work 
.'iid interpretation on all of the.se tests w^as done under the immedi- 
ate direction of T. L. Kelley. While an effort was made, through the 
assistance of T. L. Kelley and A. C. Krey, to bring to bear on each 
of ihe.se problems the help of advanced w'ork both in technique and 
subject matter, each individual contributor was free to make such| 
use of this aid as .seemed to him wdse and pertinent. As some of the 
contribution, s show, there were differences of opinion as to ihc 
value of the advice received; and in some instances there is evidence 
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of misunderstanding. The conclusions presented in the separate 
articles are therefore those of the individual contributors and must 
be so regarded. This is equally true of the statement of general con- 
clusions from the point of view of instruction in the social science 
subjects. The Commission has indicated separately its own judg- 
ment on the value of the new tyi^e test in its volume of Cxjnclusions. 

The writer of this preface would like to take advantage of this 
opportunity to express his own gratitude for the many helpful sug- 
gestions and kindly criticisms which he received on his portion of 
this volume from H. R. Anderson, Frank W. Ballou, Isaiah Bow- 
man, Edmund E. Day, M. E. Haggerty, Edna Hcidbreder, Howard 
C. Hill, Ernest Horn, Henry Johnson, John B. Johnston, E. F. 
Lindquist, Donald G. Paterson, John G. Rockwell, Joscjih R. Sirayer, 
David F. Sw'enson, Edgar B. Wesley, Ben D. Wood, Clifford VV^iody, 
and C. S. Yoakum. Ernest Horn, H. R. Anderson, and E. F. Lind- 
quist aided the writer on his preliminary as well as later draft of the 
material. Above all, the writer is indebted to Truman L, Kelley, not 
only for reading all the versions, but for his unfailing patience in 
instructing him in the more technical aspects of the problem during 

five years of close association. . „ 

A. C. Krey. 
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Chapter I 
INTRODUCTION 
A. C. Krky 

The members of the Commission have felt that the record of our 
work in the preparation and use of tests should he accompanied by 
some appraisal of such testing procedure, both as an instrument 
of investigation and as a possible aid to instruction. The Execu- 
tive CoiTimittee has assigned this task to the Chairman. 

Such an appraisal is a highh' difficult problem which can scarcely 
be solved to the complete satisfaction of any group or individual. 
It in\'olves the judgment of instruments de\'ised by psychologists 
and other workers in educational measurement for the investiga- 
tion of learning and instruction in social science subjects. Experts 
in one field of learning are forced to consider the work of experts 
in another field. 1 he learning process, one of the chief preoccupa- 
tions of psychology, must be considered as applied to social science, 
the special preoccupation of another group of scholars. A philos- 
opher might write an interesting essay on such a problem, but it 
would probably not be accepted by psy chologists and would have 
little direct value to the teacher in the social sciences. Whether a 
social scientist may undertake the task with any greater assurance 
is doubtful. 

Psychology is not as yet a definite and unified science yielding 
incontrovertible results of practical utility to other fields of learning. 
I'he problems with which it deals are many. It approaches these 
problems from a number of different premises. There are several 
different schools or systems of psychology at present, and in- 
dividual psychologists draw freely from two or even more of these 
systems.’ Ihese schools and individual workers are marked not 
alone by the particular problems of psychology which they attack 

•HcidhreiltT, E., Srifu Psycft'jloeirx, New York, Woodworth, R. S., 

Conlfnipurary of Psychology, New York, 19.^1; Higginson, G. D., Fields of 

Psychology, New York, 19.?!. 
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and the different premises from which they start but nearly as 
much by the different vocabularies which they use in describing 
their problems and procedures. All of these psychologists are con- 
cerned with some problems of interest to the social scientists, and 
some problems of social science are of interest to all schools of 
psychology. To the layman it might appear, therefore, that both 
psychologist and social scientist should be able to offer a definite 
contribution to the common task. No one, however, would at 
present feel qualified to define that contribution in such a way as 
to satisfy the psychologist and yet be of service to the teacher in 
social science. 

The writer has chosen, therefore, to present his statement more 
or less as a narrative of the work of test construction during a 
period of four years, together with such judgments as have oc- 
curred to him in watching that development, d'hc trials and 
errors as well as the achievements have been recorded, (.’onclu- 
sions reached at one stage are shown undergoing revisions at the 
next. This procedure has been followed in the hope that the 
reader, whether interested in educational measurement or in teach- 
ing the social sciences, may thus share the experience of the test 
committee and apply to the series of experiences the technical 
terms which he prefers. I bis effort will have served its purpo.se if 
it calls to the attention of workers in both fields the nature of the 
problems involved and the possibility and desirability of greater 
mutual service. 

This narrative and appraisal of our work with tests may serve 
as an introduction to the subject for teachers who, like the writer, 
are relatively inexperienced in the newer techniques of measure- 
ment. Tliose who are familiar with these techniques may find it 
more profitable to begin with the more technical material which 
follows. "Ilie determination and details of the plan devised for 
this phase of the investigation are described later.* For the pur- 
poses of this introduction we may begin with the scrutiny of tests 
available in 1929, when this work was started. 

‘See Chap. II, 
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Scrutiny of Existing Tests 

The application of the new test technique to the social studies 
was well under way before this Investigation was begun. Efforts 
had been directed chiefly to the measurement of results in the 
elementary grades. Some work had been done at the secondary 
school levels and a few special projects had been launched in the 
college field. Enterprising publishers of textbooks, recognizing in 
the new-type test a coming vogue of the educational world, had 
begun to develop tests to accompany their textbooks. 

Most of the tests available might be classed as tests of specific 
information or of definitions. Some of the tests combined the two, 
A few attempted to test “relationship,” “thought,” and “judg- 
ment.” A variety of new-type questions were applied in these 
tests, the true-false, multiple-choice, matching and completion 
types being most common. Paragraph analysis and some critical 
exercises were included under the general form of reading-compre- 
hension tests. 

A few' of the tests were standardized; that is, they had been 
tried out in an experimental procedure w'ith a sufficient number of 
pupils to afford a basis for comparison useful for the country as a 
whole. Such tests were usually published in several equivalent 
forms, thus affording an opportunity to test progress in a given 
class.* 

The pedagogical literature expounding and criticizing these tests 
indicated that the emphasis m their construction had been placed 
upon the measurement technique.® Both the reliability of the 
equivalent forms and the validity of the tests as measures of pro- 
gress rested upon the statistical showing of the results. There was 
almost no critical discussion of the tests as legitimate measures of 
knowledge as determined by those wdio had advanced far in the 
knowledge of specific subjects. 

Several months were spent in the examination of these tests by 
Dr. Kelley with the assistance of the members of the staff of the 

•See Bibliography, n his observation has recently been confirmed in a more 
technical treatment of the matter by E. F. Lindquist and H. R. Anderson: .^chieve- 
ment Tests in the Social Siudics, Eduralioml Record, April, 19.53. 
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Investigation. It was agreed that the greatest merit of the existing 
tests lay in their measurement of range of specific information and in 
their recall of specific verbal definitions. For the purposes of this 
Investigation, however, this merit was of very limited value. Pu- 
pils who had had a course in ancient history might be expected to 
know a considerable amount of specific information in that field, 
and students who had had a course in sociology might be expected 
to know the precise definitions offered in that course. What, how- 
ever, would be the value of applying such tests to pupils who had 
had neither ancient history nor sociology? Some, perhaps, but not 
enough to justify the construction of tests for every course offered 
in every school system nor the trouble involved in attempting to 
administer a nation-wide application of such test.s. The tests of 
“relationship,” “thought,” and “judgment” were found either to 
depend too closely upon specific subject matter of separate courses 
or to be too questionable in their assumptions to serve the needs 
of the Investigation. 

It became increasingly clear that the whole range of information 
and values offered by the social studies program of the country 
could not be tested within the resources of the Investigation. The 
Commission must content itself with achieving as much as pos- 
sible within its limited means. This necessity made it further de- 
sirable to focus the efforts of the testing program upon those values 
which the several subject-matter fields had in common rather than 
upon their separate contributions. The tentative list' of these 
more nearly common values as drawn up by the planning commit- 
tee became increasingly Important. Exi.sting tests afforded prac- 
tically no help in this endeavor. The relatively slight progress 
which the tests of “relationship,” “thought,” and “judgment” had 
made toward the measurement of the more intangible values con- 
tained in the tentative list of objectives was not very encouraging. 
Nevertheless it was decided to attempt tests of as many of the 
types of outcome of instruction contained in the list as possible, 
provided competent specialists in the social sciences could he 
found to undertake the actual construction of the tests. As cir- 
‘This list appears on pp. 12 vl 27 . 
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cumstances were to determine, the first of the objectives or out- 
comes of instruction listed — understanding of important ideas by 
means of which society functions — received major attention. 

I. THE KEFOR T TO TEST UNDERSTANDING 
Understanding, as the term is used by most teachers of the social 
sciences, is an intangible value over and above knowledge of specific 
information or precise verbal definitions. It is one of those values 
which nearly all teachers recognize, but few have ever undertaken 
to define. Yet for purposes of test construction it had to be defined, 
or at least identified, so that the results of tests might afford definite 
proof of the presence of understanding as defined. 

Specific information, referred to as “the facts of history,” for 
example, is distinguished as consisting of those items w'hich are 
sharply limited in time, pIace,person, event, and attendant circum- 
.stance. A student can commit a great number of such items to 
memory without attaining understanding. Precise definitions com- 
posed of certain words arranged in definite order offer the same 
difficulty. A student can commit a precise definition to memory 
without being able to apply it to any given situation. In such cases 
it becomes a mere verbalism and does not indicate understanding 
It is generally assumed that the student learns in his classes in 
social science much that w ill be of use to him as a responsible mem- 
ber of society. Obviously the specific information which he learns 
in school will all be past facts after he leaves school, whether that 
information has come to him as a narrative of an event in history 
or as a group of statistics in other social sciences. The actual life 
situations which he will face will ail be more or less different from 
those which he has studied in school. The carry-over from school 
to life, therefore, must consist of something beyond the specific 
information. It must be something about the information worked 
over in class that can be applied to other facts. Classification of 
certain facts, or relationship of facts of certain kinds, it seemed, 
might furnish the clue to understanding. And if such classifications 
or relationships were symbolized by terms, these tenns could be 
used in the construction of the new-type test. All such terms, how- 

(51 
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ever, are capable of some sort of definition and most of them are 
defined in dictionaries. Ability to repeat the verbal definition of 
such terms would not prove understanding. Ability to apply the 
term or its definition correctly would be more convincing, especially 
if the application were to a new situation not previously met. 

The distinction here attempted between information and under- 
standing may offer some difficulties to psychologists who differ 
among themselves as to the quality and value of verbal symbols on 
the printed page. Nor does the writer presume to enter into the 
vexed controversies of the epistemologist about the finer distinc- 
tions of idea, sensation, and concept. The distinction grows out of 
the materials which the social scientist uses and for his purposes is 
adopted as a practical hypothesis to be tested by results. It might 
be expressed in another way as the difference between a specific 
fact and a generalized fact. The statement that Johnny Jones was 
killed by Mr. Smith’s automobile on Brown Street, March 1, 19.30, 
is composed of a series of specific past facts having in thenuselves 
no functional value. On the other hand, the statement that cross- 
ing a street under similar circumstances is dangerous is a general- 
ized statement of fact which does have functional value. The for- 
mer may be called an inert fact, the latter a functional fact capable 
of application to new situations in the future. 

The designation “inert” for time- and space-limited facts is an 
arbitrary one. It is being made to cover a great variety of materia! 
iK^ich perhaps can not be included under a single label. Recalling 
Jrisiah Royce's definition of a community as a group of people who 
have one fact in common, even the most inert of facts might he 
used for a purpose and thus to that extent qualify as functional. 
The only justification for the use of the term in the sense here in- 
dicated is to emphasize a distinction between the materials of the 
social science curriculum, which in themselves have definite limits 
of time and place, and those materials which are unlimited. 7'his 
distinction has usually been overlooked in pedagogical discussion. 
It has been pointed out that the designation of “inert” to items of 
specific information might actually do much harm by its inherent 
implication of uselessness, whereas this material may, and, as our 
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later experience was to show, actually did, prove to be of vital use 
to the teacher. Perhaps the suggestion of one critic that “unique’' 
he substituted for “inert,” and “general” for “functional,” would 
describe the difference without any damaging connotation. At any 
rate, the terms should he ragarded as purely temporary and sub- 
ject to important modification in light of later evidence. 

It was decided to extract from the matter offered in the social 
sciences the terms expressing functional facts as the basis for the 
construction of tests of understanding. To safeguard against the 
possibility that this distinction was purely an individual or “sub- 
jective” idea, five teachers were asked to collect such terms from 
textbooks used in the social studies from grades four to twelve in- 
clusive. The distinction was explained to them and they were 
asked to work independently to determine whether such terms 
could be collected with a fair degree of agreement. I he trial showed 
that, while there were marginal differences for almost every worker, 
the lists wliich they brought back contained almost the same 
terms. 'Phis percentage of terms on which they agreed averaged 
so high that it offered a sufficient degree of objectivity to satisfy 
the requirements of the new-type test technique. 

Encouraged by this degree of success, it was decided to subject 
the whole range of public school instruction in the social sciences 
to similar analysis. I he range of public instruction was interpreted 
to extend from the first grade through the junior college, or sopho- 
more year of college. It might he assumed that a comprehensive 
analysis of the material used in college courses in the social sciences 
would yield every term expressing social relationship to be found 
in the earlier years. Nevertheless, the texts and more common ref- 
erence materials used in the elementary and secondary schools 
were also analyzed. As an additional precaution, an analysis of 
news columns of typical new'spapers was also made. This seemed 
desirable in view of rhe widespread belief that there w’as insuffi- 
cient relation between instruction and the problems of daily life. 
A comparison of functional facts of social science as found in the 
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newspapers and as found in the school books promised to reveal 
any such disparity.^ 

Limitation of the List of Functional Facts 

It soon became apparent, as the compilation of the list proceed- 
ed, that the list would grow so large as to become unwieldy. If the 
list was to be submitted to consensus appraisal, it was imperative 
that some limitation be put on it. But on what basis.? The solution 
of this problem proved to be a difficult one, but it also led to a 
number of interesting and perhaps v-aluable observations. The suc- 
cessive limitations required were usually determined after a con- 
ference with two, three, or more persons. At times the whole 
working staff of the Investigation participated. The aim through- 
out was to obtain a list® by means of which as much understanding 
of the social sciences as possible could be determined. The prin- 
ciples used in limiting the list of functional facts follow ; 

1. Requirements' of New~Type Test Technique, I he use of the 
new-type test technique, which is intended to sample as much as 
possible in as short a test as possible, imposed its own limitations. 

^ Ideas of social relationship which could not be expressed in a single 
term, or, at the most, in a brief phrase of three or four words were 
automatically eliminated. This principle of elimination had the 
practical advantage of lending itself to fairly exact application by 
any number of workers. 

Subject-matter specialists tended to resent this purely mechan- 
ical restriction, because it seemed to eliminate all the more com- 
plex ideas which could only be expressed in longer statements. 
Some of them thought that any tests constructed on so narrow a 
basis would yield only the beginnings of understanding in the field 
and leave the essential problem of genuine advance in understand- 
ing virtually untouched. On the other hand, all efforts to reduce 

‘Dr. Edith E. Ware, formerly Deatr of Russell Sa^e College for Women, devoted 
much of her rime as special investigati>r to the analysis of colleKc courses and news- 
papers. This work was further supplemented and checked b\- the late Dr. Frances 
L. Baldwin during the year m which she served a.s special investigator for the 
Investigation. Additional checks were afforded by Dr, E. K. Klli.s of the University 
of Missouri and Miss Theresa Little of the University of Wisconsin. The list as 
compiled and checked by these workers appears in the Appendix. 

®Sec complete list included in Appendi-x, 
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the larger ideas to manageable units for other than essay type 
questions had thus far failed. Analyses of textbooks and courses of 
study for either topics treated or the amount of space devoted to 
each topic had proved fruitless. Nearly a hundred such studies, 
chiefly unpublished master’s theses, were consulted in vain. The 
only manageable material which they presented consisted of spe- 
cific bits of information or exact verbal definitions. Not much 
more encouraging was the direct attack upon the larger ideas or 
“basic generalizations” made by Billings. The 888 generalizations 
contained in his list offered very few that could be used. Many of 
them were stated in terms which most social scientists could not 
accept without important qualifications. Most of those that 
could be accepted were so broad as to be practically meaningless. 
They did not, in the form in which they were stated, lend them- 
selves to the desired economy of test-construction. The course 
suggested, by our advisor. T. L. Kelley, seemed, therefore, the only 
alternative. 

Though undertaken reluctantly, this approach to the problem 
developed certain unexpected advantages. The first of these to ap- 
pear was the realization that the method adopted(yielded simple 
units of ideas, each unit capable of identification, if not always of 
precise definition, fhc larger ideas, or basic generalizations, were 
compounded of these unitary ideas. Tests constructed out of the 
unitary ideas would, therefore, thr ow important light up on the 
pupil’s mastery of the|arger y^as^f Another advantage appeared in 
the fact that tlie list contained unitary ideas which were learned 
only as parts of larger ideas. The pupil’s mastery of these more re- 
mote units would accordingly imply a much larger knowledge and 
understanding!^ To this extent, the procedure, which seemed at 
first too mechanical, yielded its compensations. It was hoped that 
perhaps the final results of such tests might afford even greater 
satisfaction. 

2. Terms Expressing Relations Among People. It was early de- 
cided to distinguish between terms designating relations among 
people and those designating relations of people to the material 
world. To include both would have made the final list too large for 
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practical purposes. In choosing to omit the terms designating rela- 
tions of people to the material world, undeniably “functionar' 
though they were, the staff was moved by two considerations. 
Besides the size of the list, it w’as felt that for purposes of test econ- 
omy knowledge of relations among people would strike more near- 
ly at the essence of the social sciences. It might be assumed that 
pupils who showed mastery of these less tangible ideas would have 
less difficulty in learning ideas which focused on material objects. 
Such tests would more nearly reveal both training and ability. 
Whether this consideration can be entirely justified or not, its effect 
was to eliminate many “functional” facts used in history, eco- 
nomics, vocational courses, and geography, leaving political science 
and sociology relatively undisturbed. The line of demarcation could 
not always be sharply drawn, especially when the term emphasized 
the relationship among people, with reference to something in the 
material world. In general such terms were retained, though con- 
sistency proved difficult and some errors have undoubtedly crept 
in, both of omission and inclusion. 

3. Elimination of the Simplest, Most Obvious, and Most Frequent 
Social Relationships. It seemed proper to exclude functional facts 
that every normal child might be expected to know satisfactorily 
before he reached the fourth grade. Such terms of social relation- 
ship as mother, father, sister, and brother did not promise to be 
very serviceable in a testing program. These terms and others al- 
most as obvious occurred so frequently that their elimination 
promised a considerable saving of time and energy, d'hcre was a 
marginal fringe of judgment here, as on several other principles of 
elimination, with the probability that errors have occurred on both 
sides of the margin. 

4. Figurative Language and Synonyms.' One of the most per- 

'This topic suggests another equally Important, the lexicographer's semantics. 
Horn, in connection with his studies in spelling and reading, had occasion to consider 
the different meanings in which /ery simple words were used. He found no less 
than eighty-six different meanings attached to the word “gcr." The extremes to 
which society can subject a single and simple word is illustrated in the term “fast," 
which regularly designates both swift motion and total lack of motion. It is possible 
that the more common words are subjected to more meanings than are the more 
learned words, though it would be diracult, indeed, to find in the New English 
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plexing problems to arise as the list grew was that of dealing with 
figurative language. I'he same idea of relationship appeared in a 
variety of forms. I his was more evident in newspapers than in 
books, more frequent in reference books than in textbooks, but not 
entirely absent from any of them. In most cases the variety arose 
clearly from a desire to avoid frequent repetition of the same word, 
however superior that word might be. The outcome of an athletic 
contest as described by a sports writer in a newspaper may appear 
as a “victory,” or “defeat,” or “rout,” or “shambles,” or “mas- 
sacre,” or “Waterloo,” or nearly all of these in the same article. 
The range of synonyms in the more sober school books is seldom so 
extravagant, but both s 3 monyms and figurative terms are used. 
More or less exact synonyms could also be classified as colloquial- 
isms, localisms, terms of sophistication, and standard terms. Pref- 
erence in the choice of synonyms was accorded to the standard term 
as having universal currency and more precise meaning. Only such 
of the others as had acquired a ver\' wide circulation and common 
meaning were included. Here again judgments were frequently dif- 
ficult and arbitrary, with the probability of error. 

5. Different Forms of the Same Word. Another source of con- 
fusion, but one which promised some prospect of elimination, was 
that afforded by different forms of tlie same word. Ideas of the 
same relationship w'ere found to appear in a definite series of forms, 
usually, though not always, variations of the same word. The sim- 
ple idea of relationship appeared in noun form. When in process, it 
appeared in verb form. When used in a comparative sense, it took 
on an adjective form. When used to designate a functional use of 
the idea, it appeared in adverbial form. When the idea had become 
sufficiently common or widespread to acquire a permanent organi- 
zation to support it and a fixed location or headquarters, it appeared 
in what might be called an institutional form, usually Indicated by 
the suffix “-tion” or “-at.” When the idea was a matter of contro- 
versy, this was often indicated by the prefix “anti-” or “pro-” or 


Dictionary words of only a single meaning. It would be most difficult to find such 
terms in the social science field. The association of a single meaning with a given 
word must, therefore, be regarded as an ideal rather than an assumption. 
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the suffix "-ist.” When the idea had come to have fixed associations 
with a more or less definite group of other ideas and attributes, this 
was indicated by the suffix “-ism.” Sometimes the word “-minded” 
or “-conscious” was used as suffix to indicate what might be termed 
the ideological form of the word. 

It appeared that the list could be materially reduced, if only one 
form of the word w’ere included. The simple form of the idea, the 
noun form, was given preference unless the institutional and ideo- 
logical forms represented so much modification of the original as to 
constitute virtually separate ideas. Thus, society, social, and social- 
ism had all to be Included. So, too, brother, brotherhood, and fra- 
ternity. It was not always easy to determine when the degree of 
difference in meaning had reached proportions necessary to include 
another word as a separate term in the list. There are undoubtedly 
errors on both sides of the line. 

6. Obsolete and Foreign Terms. It .seemed fair to omit terms 
that once served society, but were no longer used to express the 
idea. Terms in foreign languages sometimes used in English for an 
idea of relationship adequately represented by an English term 
might be characterized as sophisticated and omitted on that basis. 
Terms rarely used could also be omitted, if the idea was commonly 
expressed in another term. At least it seemed fair to exclude them, 
in order to keep the list down to manageable size. 

The effort to do this, how'ever, also occasioned its difficulties. 
There was little difficulty in eliminating such a word as “fief.” On 
the other hand, wffiat was to be done with a term like “slavery?” 
The United States had formally abolished slavery in 1863. But the 
League of Nations found millions of people still in a state of slavery 
as late as 1930. Obviously the term “slavery” was not obsolete. 
Another source of difficulty was found in the fact that many fairly 
common ideas were expressed by different terms in successive gen- 
erations or even more frequently. The margin of choice was much 
wider in applying this principle than in most of the others and the 
possibility of error consequently much greater. 

7. Hierarchy of Officials. Another source of perplexity came from 
the variations in title used to describe the hierarchy of officials in 
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the greater variety of organizations, social, economic, and political. 
The differences between the offices themselves, as ideas of social re- 
lationship, were too slight to justify rhe inclusion of all the varied 
forms by which an essentially similar office w^as entitled in different 
lands or social groups or industrial groups. It was decided, there- 
fore, to retain in each case a complete hierarchy of titles as the type, 
and to add only such additional titles as special considerations 
made necessary. In this case, too, the margin was very wide, and, 
in fact, necessitated selection between two margins, with corres- 
ponding difficulty and probability of error. 


Significance of the Final List 


The relationship of the final list to the whole body of social 
science materials may be represented in diagram form; 


More intriccta social rela- 
tionships incapable cf ex- 
pression by a sinjile term 
or brief phrase. 


Relationships among people. 


Simpie type: o( rilstionship among people. 


Rclatiorsl’iips to meterial world, 
e.g., cccupatiorrs. geographic influences. 


Simple relationships of time and place. 


Events 


Piacss. concrete objects (reatia), persons. 


--Omitted as beyond 
the range of ordin- 
ary new-type tests. 

- - Field of tests of 
understanding. 

--Omitted 


--Omitted 

--Omitted 
--Omitted 
— Omitted 


The list as finally compiled has a qualitative rather than a 
quantitative significance. No direct effort was made to determine 
frequency of occurrence. This entered into calculation incidental- 
ly as a basis for the elimination of rare, unusual, and obsolete 
terms. It may, therefore, be assumed that all the terms in the list 
occur frequently enough ro he a potential charge upon learning in 
the social sciences. 

The defect in qualitative range suggested by the application of 
the mechanical requirement of no more than a single phrase proved 
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to be less serious than at first supposed. The larger ideas in their 
ramifications include many terms which occur only in the matrix 
of those larger ideas. Many of these were caught in the list, and 
their inclusion in a test, therefore, might imply, if not fully prove, 
the understanding of the larger idea. 

Though still lacking the more elaborate and complex ideas ex- 
pressed in sentences and paragraphs, which may represent the 
most advanced knowledge, the list affords a fair sample of the 
range of ideas taught in the social sciences through the sophomore 
year in college. It actually touches a number of ideas not usually 
acquired until later in college life. 

A student who showed a satisfactory knowledge and understand- 
ing of the functional facts contained in this list might he presumed 

(1) to have studied the social sciences through the junior college, 

(2) to have an understanding of social ideas beyond the average 
adult, (3) to have capacity for advanced study of the social 
sciences. 

The inclusion of the whole range of courses offered in the social 
science field in the grades from the fourth through the junior college 
— history of all periods, civics of .several kinds, economics in sev- 
eral forms, sociology and geography, and the contents of a year’s 
run of several current newspapers — afforded a wide and probably 
unusual view of the growth of social ideas. Some of the observa- 
tions made by the several workers engaged in the task mav he of 
value, therefore, in suggesting hypotheses as to the learning pro- 
cess in this field.' 

1. The distinction between “functional” and “inert” or “unique” 
facts in social science seems to be valid in the sense that different 
workers employing this distinction independently arrived at lists 
that were almost identical. 

2. Inert or unique facts must he qualified as ranging from 
literally inert to a considerable degree of functional value, or, 
rather, inert facts can be best described as incomplete functional 
facts. 

This observation nray require some explanation. Social ideas are 

ifor a fuller discussion of this point, .see pp. 4^-5.?, lO'Al IV. 
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learned in many ways — outside the classroom as well as within. 
When learned outside the classroom, the learner fends to associate 
the idea learned with some person, place, or material object present 
when the idea dawned upon him, whether there was any such essen- 
tial connection or not. iit refers henceforth to the idea in terms of 
the object, place, or person with which he has associated it. If the 
idea is one of common occurrence and the association used appeals 
to others as preferable to the term they then use, or if it supplies a 
name for an earlier idea, it gains general currency and is well on its 
way to the status of a functional fact. It may be said to have reached 
that status when the proper noun which is usually employed at 
the outset becomes a common noun. 

Thus the idea of wanton destruction became associated in the 
minds of medieval churchmen with the sack of Rome in 4SS by the 
V'^andals. 4 he association acquired a permanence and popularity 
until toda> “vandal” and “vandalism” are common nouns indicat- 
ing that idea. Thus, too, the idea of mass production, with its com- 
plex of social connotation, has become identified in European lit- 
erature with Henry Ford, and the term “fordisme” and “fordis- 
mus” have already become functional terms in Europe. Many 
politicians today use the term “gerrymander” unconscious of the 
fact that it is derived from the name of an individual associated 
With that type of political redistricting. Wall Street has become a 
synonym for financial speculation, as firoadw ay has for theatricals. 
I hey may be cited as examples of proper nouns well on their way 
toward the status of functional terms. 1 ammany Hall as a syno- 
nym for machine politics has alread\ moved far in that direction. 
“1 amrnanyism” needs only to alter the capital letter to be said to 
have attained this status. 

3. Functional facts, or terms describing very common .social 
relations, are in constant danger of being supplanted by otlier 
terms. Doubtless many factors enter into the explanation of this 
phenomenon. 1 he fact that such terms represent extra-.school re- 
lations; the desire to vary the monotony of using the same term 
for an idea wTich so frequently recurs; an unconscious feeling of 
incongruity about applying a “book” word to a life situation, as 
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well as the tendency to describe the situation in terms of associa- 
tion, all play a part. Every generation, it seems, adopts its own 
terminology to describe courtship and its associated ideas. A lexi- 
cographer could be kept busy listing the various expressions that 
have been devised to describe direct money grants by Congress 
alone. 

4. There seems to be an increasing order of difficulty in the vari- 
ous forms of an idea as expressed by different forms of the word. 
Only the simple idea in noun form registers a high degree of accu- 
racy and precision in use.' Next in simplicity to the noun apparent- 
ly comes the verb form. The adjective follows, but the adverb is 
much less used. The institutional form represents a fairly high 
degree of learning, while the ideological form is seldom used accu- 
rately. The basis for this observation is the frequency and varying 
connotations with which these forms were found both in news- 
papers and in textbooks. 

5. Only a relatively small proportion of the functional facts of 
social science may be regarded as definite and permanent, fhose 
terms which are used only by the scholar or the advanced student 
have the best chance of retaining a precise meaning. Those which 
serve the whole population seem to be in constant danger of modi- 
fication. This observation is most strikingly confirmed by the fate 
of terms coined by scholars and usurped by the general public, as, 
for instance, those connected with psycho-analysis. It is likewise 
illustrated by the recent efforts of scholars to state some of their 
abstract findings in simple language. 

Difficulties Encountered in Evaluation of Items 

It soon became apparent that the preparation of the comprehen- 
sive list covering all social science w'ork through the junior college 
years would require a large amount of time. I'here w as also strong 
reason to believe that the further step in the preparation of the 
tests W'ould require as much, if not more, time. Under the circum- 

*Thc relative difficulty of the simpler forms is not so strongly marked. (Jrtati r 
vividness or concreteness or more frequent u.se may in some cases reveal a verb 
as more easily learned than the noun and, in fewer instances, even the adjective 
and adverb. In general, however, the order of difficulty appears as stated. 
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stances, it seemed wise to use the original list compiled from text 
and obvious reference books used between grades four and twelve 
as the basis for preliminary test work in understanding. This list 
of some three thousand terms was sent to thirty college teachers of 
recognized scholarship and extended teaching experience with be- 
ginning classes in the social sciences in all sections of the United 
States and in all types of educational institutions. 

These judges were asked to grade each item on the following 
bases — whether they expected the graduates of high schools en- 
rolled in their courses to sliow (1) complete understanding of the 
term, (2) some understanding, O) no understanding. The judges 
were also asked to suggest terms not on the list which they thought 
should be included. For convenience in checking, the list was pre- 
pared in alphabetical, rather than in logical, form. 

I'hese instructions, as indeed would any instructions, invited 
differences in evaluation. I'he instructors Inevitably judged a term 
on the basis of its usefulness in the courses they were conducting 
with freshmen classes. Some terms essential to a course in Modern 
History might not prove necessary in teaching American Govern- 
ment. Similar differences in evaluation were inevitable in other 
fields of social science. Hence terms absolutely essential in one 
instance might be designated as of no value in another. 

Later, when the lists in special fields of social science were pre- 
pared, similar difficulties of evaluation were discovered. Classified 
li-sts in economics, political science, and sociology were submitted 
to experts in these fields. This procedure proved unsatisfactory? 
because there were marked differences in classification followed by 
different specialists in these fields. I'hese differences were so 
marked that an expert accustomed to one form of classification 
found it practically impossible to evaluate the items in a different 
scheme of classification. The method had to he abandoned for the 
more objective and less controv’crsia! alphabetical arrangement. 
Even this, however, presented difficulties. Specialists, especially in 
the field olf sociology, were found to have a different nomenclature 
for ideas wliich laymen might regard as practically the same. This 
diversity was due to differences in point of approach. Judgments 
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of the value or importance of terms which sociologists do not gen- 
erally employ varied even more than did the judgments of the col- 
lege teachers concerned with freshmen courses. While sociology 
has been cited as presenting this difficulty, it proved scarcely less 
true in the fields of economics and political science. 

Another difficulty encountered in the evaluation of the list ap- 
peared most pronounced in dealing with political science. While 
there was much less difficulty about variety of terminology in this 
field than in either sociology or even economics, the mechanical 
limitation of confining the ideas to single words or brief phrases 
proved an almost insurmountable obstacle. 1 he specialist in politi- 
cal science found himself halted by wondering in what connection 
the term was to be used. It took on a different value as it appeared 
in different connections. Most political scientists, it became evi- 
dent, would face such a task with a sense of complete frustration. 
Only a very few political scientists, and those most neaily con- 
nected with teaching in the lower schools, could he persuaded to 
disregard their scholarly demand of qualifying connections even to 
attempt the task of evaluation. This difficulty was most pro- 
nounced in the field of political science, hut it also occurred in 
economics and sociology. 

Still another difficulty arose from the size of the r.isk it-self. Des- 
pite all eliminations, the list w'as still a formidable one. 1 he list 
for the grades four to twelve contained nearly three thousand 
terms, while the combined list for the social sciences in the junior 
college grades was even longer. An accidental omission of a page 
of the mimeographed copy of the first list revealed thi.s difficulty. 
The task of judging each of three thousand items involved not 
merely the evaluation of the separate item but its evaluation with 
reference (1) to the demands of the whole range of a year’s work, 
(2) to the pre.sence or absence of other terms in the list which more 
nearly satisfied those demands, (3) to a possible omission which 
might more nearly satisfy the judge’s needs. Since each of the 
judges was abundantly occupied with his own immedjate work, 
weighing the value of each of these terms in this circuinspect and 
C0mprehensive manner was virtually impossible. I'his was fully 
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proved by the fact that only one of the thirty judges noticed the 
omission of a page containing terms beginning with the letter “s.” 
The one person to notice the omission was the only one of the 
judges who indicated any omissions. "1 he evidence of fatigue as 
well as difficulty in finding time to finish the task was revealed by 
the fact that in several cases the terms on the last page or two 
were not checked. 

In view of these difficulties encountered in the evaluation of the 
terms, the application of a rigid statistical technique to give a con- 
sensus evaluation or weighting to each term in the list would .seem 
rather the compounding than the obliteration of error hv the com- 
bination of pidgmenfs. 1 bis was done, however, to satisfy the de- 
mands of objective teebnitjue, and, after the weightings were statis- 
ticallv established, tiu- terms were arranged in ipiartiles accordingto 
their weightings for importance. Inspection of the results seemed to 
indicate that the more rarely used and the most difficult were in 
the upper quartile, while the mttst essentia! appeared to he those 
easiest or most commonly used. Following this inspection as a 
clue, it \sas deciiied to construct test.s chief!} from the two middle 
quarters. 

( 1 riKS Mrr i.n Co.n.stri ct.'on or 1 i-srs 

'I’hi.s weighted list of suh-collegiate functional facts made it pos- 
sililc to begin the work of test constructi(>n in accord with the 
principles estahlislu d for this work. Fven in rhis stage of the work 
Dr. Kelh}' insisted uptui a combination of subject-matter special- 
ists and psN chological technicians, binding persons capable of con- 
structing test.s prosed more difficult than had been anticipated. 
Finally Dr. h'dgar H. W esley, head of the department of social 
studies in University C'ir\ High School of University City, Mis- 
souri, Mi.ss Mary Gold, head of a similar department in the Uni- 
versity of Minnesota High School, and Miss Mary G. Kelty, 
formerly in charge of thi.s department in the Oshkosh Normal 
School, and Dr. l.uella C. Pres-sey of the department of psychology 
of Ohio SlAte Univer.'^ity were engaged to construct these tests. 
All had demonstrated inttre.st and ability in this work. Dr. Pressey 
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approached the problem from the more objective standpoint of a 
psychologist long interested in vocabulary studies, especially her 
own covering elementary and junior high school levels, while the 
others represented primarily the interest of the subject-matter 
teacher. Dr. Wesley was primarily interested in the senior high 
school, Miss Gold in the junior high school, and Miss Kelty in the 
intermediate grades of the elementary school. 

This work of test construction also proved to be very time consum- 
ing. The detailed account of the construction of these tests is told 
elsewhere.^ It will be sufficient here to summarize the experience of 
those engaged in the making of the tests. Hundreds of questions 
were prepared. A variety of forms were used, the true-false, the 
completion, the matching, and the multiple-choice forms being 
most favored, i'he questions were tried out and the results statis- 
tically analyzed. After a series of trials, the multiple-choice form 
of question was selected as best for testing understanding. 

The completion type of question, which gave the student wide 
freedom in supplying missing terms, phrases or clauses or defini- 
tions and illustrations of ideas or examples of various kinds, proved 
most revealing to experienced teachers. The choice of answer in 
many instances revealed how far the student was from the desired 
understanding of the idea involved. Unfortunately, however, the 
variety of answers was so great and represented so many degrees 
of understanding that only an expert in the knowledge of the sub- 
ject would be able to grade such answers satisfactorily. It was 
found impossible to construct a scoring key for such questions 
which an ordinary clerk could employ. Despite the high diagnostic 
value of this type of question, the fact that it required experts and 
consumed more time in scoring made it too costly for our purposes. 
It was abandoned with regret. 

The true-false type of question, while apparently easy to con- 
struct and easiest to score, presented other difficulties. In the first 
place the ease of construction was apparent, not real. Very few 
questions in social science can be answered with a categorical yes 
or no, right or wrong. 'I'o construct questions which eould be so 
^See chapter HI. 
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answered proved to be most difficult. Secondly, these questions 
lent themselves too readily to pure chance performance. Incorrect 
answers might prove ignorance, but correct answers might repre- 
sent mere chance. The statistical procedure of multiplying the 
number of questions so as to overcome the factor of chance seemed 
too cumbersome for general use. This type of question was retained 
by two workers - by one who had found through years of trial a 
few questions so worded as to catch the Interest and challenge the 
knowledge of junior high school pupils, thus eliminating the chance 
element, and by another who, by using diagrams and pictorial illus- 
trations equally challenging, also overcame the chance element. 
Both of these workers, however, were extraordinarily able and had 
arrived at these questions through years of experience. They had 
managed to find questions which pupils could, hut would not, 
answer by mere guessing. Under ordinary conditions, however, 
this type of question proved least satisfactory. 

The matching type of question in which a number of related or 
comparable items are placed in two columns, the student to place 
before the items in one column the number of the related or com- 
parable item in the other, proved somewhat better than the true- 
false or completion types. 1 he question involved some degree of 
relationship, which wa.s desirable, and it could be scored almost as 
easily as the true-false form. However, it afforded considerable op- 
portunity for guess-work - too much if the columns were of equal 
length. Ciuess-work was cut down somewhat by making one col- 
umn longer than the other, thus including some items that did not 
fit. There was a further difficulty in that the mechanical labor of 
answering such questions was greatly increased by lengthening the 
columns. If the columns were very long, the time required to 
answer the question was too great, the eye-strain too severe; if 
they were short, the opportunity for mere guess-work was too 
great. Though useful, this type of question w'as not as satisfactory 
as the four- or five-option multiple-choice type. 

The multiple-choice type permitted the use of phrases as w^ell as 
of single terms. It afforded opportunity to test relationship as well 
as recall. By the use of four or five options the chance factor was 
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cut down to more controllable proportions. The options in the test 
could be gauged to a relatively fine discrimination of the student’s 
knowledge. Each item was sufficiently brief and compact to be an- 
swered w'ithout unusual eye-strain, and the answer could be scored 
quickly and accurately with a scoring key. Both Mr. Wesley and 
Miss Kelty finally used this form alone, as did also Mrs. Prcssey, 
who was working independently. 

In the course of determining the most efficient test form, it be- 
came increasingly apparent that the value of the questions depends 
less upon the form than upon the content, d'he mechanical advan- 
tages of the new-type question may easily be completely vitiated by 
careless selection of items. In fact a casual, careless, tlioughtless 
test in the new-type form may prove not only unsatisfactory as a 
test but may impair or defeat good instruction. This danger has 
been generally overlooked by test-makers. A careless cs.say-type 
test at least affords the student a chance to display his knowledge 
and ability by the way in which he develops his answers. A nev.- 
type test, on the other band, limits the student to what is set before 
him. If that is bad, he has no recourse. I'he advantages of the new- 
type test are too largely mechanical. I'he value of the test as a 
measure of achievement depends upon the matter cast in that form 
as well as upon how it is cast. 

The wording of the questions proved even more exacting than 
the choice of forms. The vocabulary of pupils at different grade- 
levels proved to be a conditioning factor of the utmost importance. 
Wording that seemed awkward or even childish to college students 
was frequently necessary to test the degree of understanding of the 
idea possessed by the pupil in the elementary or even junior high 
school. Not infrequently questions were condemned by college 
teachers, only to have the statistical analysis of results in the lower 
grades seem to prove that the questions w'ere actually testing the 
pupils of the lower grades with apparent accuracy. 'I'he more pre- 
cise and universal vocabulary of adult levels was not familiar to the 
pupils of the earlier grades. At any rate, it afforded an uncertain 
approach to their understanding of the ideas involved. 

The range of the options used in the multiple-choice form proved 
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also to be a matter of concern and importance. If only one of four 
or five options bore on the question, while the others were very 
remote, the pupil’s .selection of the correct option did not neces- 
sarily prove any real understanding. Often only the* vaguest im- 
pression of as.sociation between the question and the correct option 
was necessary to gain credit for understanding under such circum- 
stances. 

PHOni.hMS RaISM) in THF SiLF.CTlON OF QUESTIONS 

1 he questions as draw n up by Wesley, Gold, Kelty, and IVcssey 
were submitted to the inspection of at least three experts in the 
knowledge of the subject. 1 his inspection resulted in the elimina- 
tion of .some questions because of errors of fact, impression, or 
statement. 1 he maker of the test was given the opportunity to re- 
view these judgments and in some cases, notably in the case of 
questions made our for elementary and junior high school grades, 
the te.st makers insisted upon retaining some of the objectionable 
questions, it was their contention that the questions, though awk- 
wardly worded and perhaps objectionable on the adult level, af- 
forded a truer approach to the degree of understanding possessed 
by pupils at the lower levels. Fortunately .several of these que.s- 
tions were retained fur the actual trial of the tests. 

\N hen the tests had passed this inspection and had been scrutin- 
ized by Dr. Kelley tor technical form, they were tried out under 
fcpical .school conditions. As indicated in the detailed account of 
the Wesley test,* nearly every one of the tests went through a series 
of trials. For the final trials the tests were given to pupils in differ- 
ent part.s of the country, and, wherever possible, in a variety of 
school s\ .stems. I he rests were sent to Dr. Kelley under whose 
dirccfhm tiiey were .scored and the results submitted to statistical 
analysis. Statistical profiles were worked out for each item and 
rlu:.se were then submitted to the advisory Committee on Tests 
with Dr. Kelley’s recommendations for each item. The effect of 
this process of checking and re-checking the tests was to cut the 
original list of approximately four hundred items in the Wesley 
test to about one hundred and sixty, 

*8cc pp. 21‘*-227. 
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The use of a rigid statistical technique in selecting the items to he 
included in the final test was not accepted without misgiving by the 
subject-matter specialists. They saw some questions which to them 
seemed among the best testing items dropped, while others rela- 
tively poor were retained. Items were selected on the basis of pro- 
gressive performance grade by grade, the item showing the stead- 
iest improvement apparently being regarded as the best. This re- 
sulted in the elimination of some questions which most teachers 
would regard as unusually good ones and the inclusion of others 
which seemed relatively poor questions. Furthermore, since the 
statistics were obtained from a w'ide range of pupils and school sys- 
tems, what relation did they bear to instruction ? The school sys- 
tems were offering different courses in the same grades. The classes, 
especially in the later grades, included pupils who had taken no 
courses in social science for two, three, or even four previous year.s 
as well as others who had taken courses every year. The question 
whether a test constructed on this statl.stical basis under such con- 
ditions would measure anything but the growing maturity of the 
pupils was very seriously raised. Did not this procedure tend to 
wipe out all evidence of instruction? If it did, of what possible 
value could it be m the measurement of instruction? 

The only qualifications in extenuation of this procedure which 
the teacher could see at this time were: (1) the fact that the ques- 
tions used were all drawn from social science subjects, and even if 
the test measured growth alone, it would he measuring social 
growth; (2) that all the pupils tested in the trials had received 
some formal Instruction in social science, and even those who had 
omitted it during most of their high school years w’ere receiving in- 
formal instruction from other sources during that rime; (3) that the 
application of the final test, apparently one of social grow th alone, 
to instruction might show that different courses or different teach- 
ing procedures might vary in their contribution to that social growth. 
If so, such differences might have a far greater significance than if 
gleaned from a more definitely instructional test, whose value still 
remained to be determined. But these were all questions that would 
require later answer. With these misgivings, but with a determina- 
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tion to follow oitt the dictates of objective measurement, neverthe- 
less, the final tests w’ere drawn up on that prescribed basis. 

SUPPLEMENTARY .STUDIE.S IN TESTING UNDERSTANDING 

FIXPI'RIMENT ON ACQUISITION OF “InERT” AND ‘TuNCTIONAL” 

Faces by High School Students 

Fortunately, other workers were developing rests during these 
years, and the Chairman of the Commission had the assistance of 
a seminar, many of whose members were particularly interested in 
this problem. With their help many of these questions were at- 
tacked and some suggestions toward their solution, if not the final 
solutions, are possible. Four projects undertaken by members of 
this seminar at the University of Minnesota and the work of Lind- 
quist and .Anderson at the University of Iowa may be taken as 
examples of this supplementary work. 

T he first project was an experiment to determine the relative 
learning of “inert” or “unique” and “functional” facts. This has 
been reported in full elsewhere' and may he summarized here. 
Modern history was chosen as the subject-matter field, partly be- 
cause it is unusually rich in “inert” or “unique” material, predomi- 
nantly political, and partly because it is generally regarded as diffi- 
cult. -A course in modern bi.story is offered in the second year, tenth 
grade, of high school and again in the first year of the university. 
I'wo groups of seniors in the University of Minnesota High School 
who had not elected modern history were u.scd for the purposes of 
the experiment. Their performance was checked hy comparison 
with tenth grade classes in the University High School and in one 
of the Minneapolis high schools, as well as with a section of the 
university cla.s.s in modern history. Ihe first experimental group 
was given a week in which to work up the whole field of modern 
history. I'hough given the opportunity to obtain assistance from 
experienced teachers, they chose, after the first two days, to do all 
the work themselves. They had available three of the most widely 

'Krcy, A. C., and VVrdcy, E. B., the New-Type IVst M«?asu(r Rt suit ..' 
Hittorirai (httlmkt XXIII (January, 1932) p. 7. 
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used textbooks in modern history, including the one used by the 
pupils in the tenth grade, and, for further reference, one or two 
textbooks used in college. The same procedure was followed with 
the second experimental group of University High School Seniors 
three years later. 

For the first test, a group of experienced teachers enrolled in the 
seminar, went through the high school textbooks and constructed 
the examination on the principle of covering the whole field, using 
important items rather than obscure details and avoiding questions 
which required a new or original rearrangement of the information. 
The new-type forms of questions were used in the test, d'he test 
was submitted to a number of experienced teachers for criticism 
before it was applied. For the second test, Dr. Wesley, assisted by 
M iss Mary Gold and Miss Dorothy Bovee, selected “functional” 
facts which were especially emphasized in modern history, terms 
dealing with international relations, comparative governmental and 
social institutions. Fhey used the multiple-choice form of the new- 
type examination. Both tests were applied simultaneously to the 
university group and to the two high school groups. The scores 
follow: 

TFST OF ‘•FUNCrioNAL” F.ACFS OR UNDFRSTANDINXi 
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Lcperimetual 
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SfifctcJ 
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xxxxx 
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•KXVXXXX 

70 

X 

X 

XX 

XX 

CS 

XX 

XX 

XXXX 
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XX 
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55 

XX 

XX 
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TEST OF “INFRT" 

FACTS OR 

INFORMATION 
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‘After this pupil ctiiiipleted the tcit it was ivariicd that he had had a Modern 
History cmirse ui another Sfhiml. 


I nirrprrtcilion of the Results. Since rhe number of pupils was 
small, only tentative inferences may be drawn from the experi- 
ment. These, however, may be listed as follows; 

(1) I'he experiment lends support to the conclusion of the special 
investigators compiling the lists of functional facts, that(there is 
an impf>rtanf difference between inert, or unique, and functional 
facts in social science) 

(2) M hat inert, or unique, facts are much more readily learned 
than functional facf.s^ 

(3) [That an examination or test constructed chiefly of inert, or 
unique, facts does not measure instruction in the subject. Such a 
test can be satisfied by cramming) 
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(4) 7'hat an examination or test constructed chiefly of applied 
or differentiated functional facts seems to require something be- 
yond cramming and reflects sustained instruction or reflection, or 
both. 

The second project to be considered here was a study of the 
results obtained on the Wesley-Gold test by all the pupils of a 
junior-senior high school. The classes in these six grades averaged 
about 100 pupils per grade. Careful I. Q. ratings had been estab- 
lished for every pupil in the school. While nearly all of the seventh 
and eighth grade pupils were taking some course in social science, 
a number of ninth grade pupils, nearly two-thirds of the tenth 
grade, one-third of the eleventh, and about half of the twelfth 
were nor taking work in this field at the time of the test. This 
project thus afforded some opportunity to compare the results on 
the test of those who were and those who were not enrolled in 
social science courses in each grade, as well as to compare the re- 
sults with the I. Q. rating of the pupils. 

As expected, there was a definite improvement in the scores 
registered by each successive grade up to the twelfth. Pupils not 
enrolled in a social science course at the time of the test showed 
improvement, as also did those enrolled. A comparison of the rest 
scores of those who were and those who were not taking the con- 
current course in social science revealed several points of difference. 
When the relative rank of the pupils in each class as determined 
by their I. Q. ratings was used as the standard, those who 
were taking concurrent work in social science improved their rank 
order on the Wesley-Gold test, while the others fell below their 
I. Q. rank in every grade except the tw’elfth. The margin of im- 
provement was greatest in those years in which some history 
course was taken, the improvement being equally marked whether 
the field was ancient, modern, or American history. Again, as in 
other trials of the test, pupils in the lower quarter of the L Q. 
rating registered the greatest inprovement in those grades in w hich 
history was the concurrent social science study. 

The fact that at the twelfth grade there was no difference be- 
tween those cnrollexl in the social science course and those who 
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were not was puzzling. This might seem to indicate that the test 
measured a social growth which had been attained at that age 
without reference to instruction. Or it might mean that the test 
had reached its ceiling hy that time. Investigation, however, re- 
vealed that in this particular high school all pupils were required 
to take either the eleventh grade course in American history or 
the twelfth grade course in social science. The comparison of 
results on the test at the twelfth grade was, therefore, a comparison 
of those w'ho w-ere taking social science concurrently with those 
w'ho had had American history the year before. 

Kkfort to Dkterminf. Relative Contribution of Different 

SuBJECT.S to KnOWT.FDGK OF FUNCTIONAL FaCTS 

The third project undertaken by members of the seminar was 
the trial of the Wesley-Cold test under ordinary school conditions. 
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Miss Prudence Cutright and Mr. Walter Anderson, both members 
of the Research Bureau of the Minneapolis Public Schools, tried 
this test on a sampling of at least two hundred pupils of every 
gracie from the sixth to the twelfth, inclusive. I'he test was ad- 
ministered near the close of the school year, thus affording some 
evidence on what influence, if any, the current work in social science 
might have upon the scores. (See table on page 29.) 

The small gain in the 9th grade, in which Community Civics 
was the course offered, was disturbing. More of the terms used in 
the rest, the ideas involved, were mentioned in this course than 
in any other except the I2th grade course in social science. \'et 
every other grade except the last showed a greater gain than did 
the 9th grade. The most significant gains were made in the Amer- 
ican history course of the 8th grade and the modern history course 
of the 10th grade. Fearing that the showing of the 9th grade 
might have been due to some disturbance caused by the fact that 
this grade marked the end of the junior high school, the test was 
applied to 9th grades working under other conditions. neigh- 
boring rural school system following the older 8-4 plan of organi- 
zation tried the test. Here the 9th grade showed a marked im- 
provement, but the course offered in that year was ancient history, 
not community clvic.s. The University High School offered another 
check. It, too, was organized like the Minneapolis schools on the 
6-.3-.1 plan, but, unlike that svstem, it offered ancient history in 
the 9th grade Here again there was a marked gain. 1 he explan- 
ation of the difference would, therefore, seem to lie in the course 
itself, or in the w'ay in wTich it was conducted. 

Another effort to check these results was made with the help of 
the St. Paul school system. Here the course in community civics 
was offered through both 8th and 9th grades, two years being 
devoted to the work instead of one. A sampling of about one 
hundred pupils for each grade from the 7th to the 12th revealed 
a somewhat greater gain for the work in community civics, though 
even there the gain registered by the history courses was greater, 
American, ancient, and modern history indi.scriminately showing 
the greater gain. 
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In order to check the showing of the 9th grade still further, Cut- 
right and Anderson constructed a test based directly on the work 
offered in the course as conducted. They Included in their test a 
number of questions prepared by the author of the textbook. I'hough 
making an effort to keep their test one of understanding rather than 
of information recall, they included a number of definitions as 
offered by the text and derived their functional fact items from the 
information offered by the course. 1 he test was administered to 
another sampling of pupils from the seven grades. The results 
follow; 


Scon- /«-\ 7 \ KA '>11 '>\ lOH liB llA 12A 


72-71... 12 1. 

...... . . 2 2 4 2 

1 7 ^ 2 2 2 

rj-65 ^ 10 - 0 >, '> 

>0/)2 T' ') 17 '' 20 
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3 2' 22 !'* 7 7 K 

q-33 1 '< U H 10 6 .s 10 

4S-30 1 7 2‘> 12 31 7 10 9 

4.V-47 2 17 2s 10 17i 3 7 2 

42-44 1 2 11 21 9 20 .' 6 (> 

.39-41 1 10 13 27 3 H 2 3 2 

3f'>~3S n 12 IH 2 6 .. 4 1 

3.3-33 2 3 13 1<> I 2 1 

30-32 (< 13 i<> ‘‘I I .3 

27-29 >. 13 11 4 I 

24-26 4 2 10 2 . 2 

21-23 7 .3 4 1 1 

18-20 4 2 2 I 

13-17 6 4 1 

12-14 2 1 


Total 39 72 1 3'> 240 107 p,? 39 R6 

Median 24 4 32 1 37.9 47.3 ;3 0 .30 4 37 4 .31.4 37 3 

Gain from Grade 

toGrade 7 7 3,S 17.1 -4 6 2.4 -3,6 2.5 


These results showed the greatest gain in the 9th grade, the gain 
as registered at the end of the 9th grade being greater than the gain 
made during the previous two \e.ars. It remains to he determined 
what significance may l>e attached to the fact that, though all the 
pupils tested in the later grades had had this same course in the 
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9th grade, there was an actual loss in two of the grades and no 
appreciable gain during the following three years. 

Another project which has some bearing upon the test of under- 
standing was carried on by Cutright and Anderson, together with 
Mr. B. Lundquist, instructor in geography at the University High 
School. The Parker-Calkins l est of Geographical Understanding, 
Parts I and II, Form 1 was applied to a sampling of pupils in 
six grades, with the foilowdng results: 
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These results indicated their own problem, the determination 
of the cau.ses for the greater ga n in the two history courses, grades 
8 and 10, than in the course in geography, grade 7. Lundquist ad- 
dressed himself to this problem. A check trial of the rest in the 
University High School showed sub.stantlally the same result. 
Analysis of the items in the test and their behavior in the two 
trials afforded very little light. Analysis of the course in geography 
offered in the two school systems as well as the methods of pro- 
employed yielded no convincing explanation. One conjec- 
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ture, that the ideas acquired in the geography class demanded 
maturing and would not register until the following year, was 
entertained. This might explain the greater gain in the year 
following the w'ork in geography- It did not, however, explain why 
there was no appreciable gain in the second year following, in the 
course in community civics, and why there was such a marked 
gain in the third year following, when modern history was offered, 
fhe only other inference suggested w'as that the geographical ideas 
acquired greater meaning as a result of the vicarious experience 
with those, ideas which the narrative history courses supplied. No 
positive proved explanation was found. 

Learning of Organized Information 

.All of these projects carried some implications as to the ways 
in which children learn about society.' A closer analysis of some of 
the details yielded further implications. In the trial runs of the 
Wesley Test the term “sepoy” was used. This item was one of 
those which, according to the principles set forth earlier in this 
introduction, should have been eliminated. Unimportant in itself 
and easily learned, it seemed absurd to include it in the test. 
Though marked for omission as unessential, a typist’s error per- 
mitted its survival. On statistical analysis of the trial results, it 
proved to be one of the most discriminating items of students’ 
knowledge and interest in the social studies. Why.^ 

While this question was still unsolved, some suggestions of an 
answer appeared from apparently unrelated quarters. About this 
time one of the investigators observed two small boys aged eight 
and ten engaged in building a locomotive. Their “w'orking draw- 
ings,” the result of much planning, were nailed to a tree and the 
materials for the enterprise, accumulated from various parts of the 
neighborhood, were piled up about it. With a four-wheeled wagon 
as the base and an empty oil drum as the superstructure, the pro- 
ject was well under way. An engine cab neatly constructed of 
broken laths was attached to the rear. Various cans were soldered 

A 
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on the top of the oil drum. One was attached in front and a light 
was installed. A piece of pipe was attached as a chimney, and. 
after some days, the finished “locomotive” was moving about the 
neighborhood. The investigator recalled the sustained interest of 
these boys in locomotives. They had seen thousands of locomo- 
tives, had visited round-hou.ses, clambered all over the old dis- 
carded engines and had even had the thrill of being shown over one 
of the most modern engines by a real engineer. He also recalled 
that these boys had been quite content to play vsith toy locomo- 
tives which represented little more than impressionistic sbape.s- - 
bad preferred them in years gone by to the more elaborate imita- 
tions. It was therefore doubly interesting to compare their ideas 
of a locomotive at the ages of eight and ten w ith their itleas of the 
same subject at four and six. d'he difference consisted primariK in 
the additional detail w’hich they now demanded. Vet even at eight 
and ten, the elaborate motion gear without which a locomoti\ e is 
not a locomotive at all was still missing. The “working drawings" 
provided for a c', Under and there was also a rudimentary indication 
of at least one driving rod, hut practical difficulties had prevented 
their inclusion In the finished “locomotive." 

In Miss Van Bibber’s study of specific difficiiltie.s in tlie Teaching 
of the social studies as indicated by teachers of those subjects, the 
Federal Reserve system was one of the topics most frttijuenrly 
mentioned. Teachers from ail parts of the country and from all 
grades between the fourth and twelfth were included in her .study. 
She was struck by the fact that in the ca.se of some of the topic.s, 
notably the Federal Reserve system, the difficulty increased as the 
students advanced. Only a small proportion indicated it as diffi- 
cult in the seventh grade, whereas the proportion of teachers re- 
porting it as difficult .seemed to increase wUrh each grade, being 
greatest in the twelfth grade. 

The connection hetw’een the “sepoy” item, the .steam locomo- 
tive, and the Federal Reserve system is not very obvious, yet there 
may be a common element, i he term “sepoy” gains prominence 
from the “Sepoy Mutiny,” a detail of the history of British admin- 
istration in India which is treated as one of the topics of modern 
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history. Only tne student who had thoroughly studied his modern 
history and had been interested in problems of governmental ad- 
mini.stration would he likely to recall the episode. Hence the fine 
discrimination which the item exercised in the test. Growth in 
knowledge of the steam locomotive was likewise evidenced by the 
ijtcreased knowledge of detail. I’he increased difficulty in under- 
standing the Federal Reserve system was doubtless due to the 
greater detail which was involved in its consideration in the upper 
grades. .All three items represent highly organized information, and 
their understanding suggests the observation made by William 
James that “a person can .sec into a generalization onlv so far as 
his knowledge of its details extends.” 

.A truer test of the pupil's understanding of such highly organized 
information is furnished by his knowledge of the relationship of a 
few scattered details to the whole than by a de.scription of the 
whole. I'his is in accord with the popular observation that a 
person most advanced in understanding of a game is marked by 
bis knowledge of its fine points, and games, too, are organized 
bodies of knowledge. No need to ask a person who knows a "hit 
and run" play whetlier he understands baseball, nor one who 
knows the “V'anderbilr convention" whether he understands the 
game of bridge. For the .American pupil the "sepoy" item 
marked one of the finer points of governmental administration and 
modern history. 

Dr. Francis F. Haldwin, in working over the returns on the ex- 
pert appraisal of the functional facts in economics organized some 
of rlie topics in terms of increasing difficulty.' I he topic of banks 
and hanking may serve as an illiisrranon.- Hanking is an institu- 
ti HI <if modern societv with winch every adult has some acquaint- 
ance. A glance at thi-s array of functional facts connected with 
hanking might enable each individual to determine the degree of 
hi.s understanding of the in.sritution. d he difference between the 

'Pcthaiw “increasing kimwledgc” wuultl better describe the quality of the 
ariangemeiu. Some <tf the minor items included among the more “difScuU” are 
not in ihemsitves diffirnlt, but they would not usually c<inie writhin the range of 
the stiuient until he b.-»d learned much about banking, 
following chart. 
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expert and the man in the street is determined at the right of the 
diagram, not at the left. It is unfortunate that the same terms, 
“bank” and “banking”, summarize this knowledge, for it permits 
the man in the street to cherish the illusion that he has a full under- 
standing of the institution. In fact, the less he knows about it, the 


CHART SHOWING RATING OF TERMS USED IN 


Check 

Currency 

Loan 

.Silver dollar 
National bank 
Interest rate 
Bank<' .Assets 

Building and Loan 
Association 
Draft 
Auditor 
Fostai savings 
bank 


I Gold biiilltan 
Silver bullion 
Reserves 
Branch bank 
Monetary unit 
Silver certificate 
l.iold ccttificate 
National bank 
I system 
(iold standard 
Balance sheet 
Credit instruments 
.Mortgage loan 
Ranker's bank 
Certified check 
(>e[>osits as cur- 
rency 
Call loan 
Collateral loan 
National banknote 
Acceptance 
Chain bank 
.Greenbacks 


United Stales national 
debt 

Deprivation 
Federal Reserve bank 
Federal Reserve note 
Federal Reserve bank- 
note 

Money standard 
Lawful money 
Token money 
United Stares hanking 
system 

Hank accounting 
Discount of tommettial 
paper 

Credit money 
Easy money 
Treasury note 
• Note issue 
.Subsidiary com 
U. S. Federal Reserve 
Board 
Call money 
Fiduciarv money 
Federal Reserve system 
Representative money 
Rediscount 
Rediscount of eom- 
merrial paper 
Free coinage 
Gratuitous coinage 
Limited coinage 
Letter of credit 
Cheap money 
Treasury loans 
Federal Land Bank 
I Membet bank 


MONEY AND BANKING 

f Mutual savings bank 
Bill of eschange 
U.S. regulation of interest rate 
Special interest acioiml 
Private bank 
.Stare bank 
State savings bank 
Clearing house 
Dear nionev 
Warehouse receipts 
( <>lict rion.s between banks 
Ouaiitity theory of money 
Index number 
Mint par 
Mint charge 
•Vfinl value 
Brassage 

Clearing house reriiftcate 
Foreign rstbangc 
Rate of exrhange 
Specie leserves 
' Siaiidardiscd dollar 
siabitired dollar 
Bank for interest setllemenli 
Bimciallisin 
Elastic curicncv 
(irrsham's law 
Seignorage 
Limping standard 
Arbitrage 

Debasement of curretuy 
Par of eschange 
.Secondary reserves 
Velocity of circulation 
Mobilisation of credit 
('oniraciion of curicmy 
Infiation of currenev 
Purchasing power par of cs 
change 

Centrali/ation of reserves 
F.(|uarion of cschatiir^e 
International Sow of gold 
(Gold exchange standard 


DEGREES OF UNDKR.STANIHNG OF BANKlNfJ 

Average child Bank at a place where money is tafegiiarded. 

" youth Bank at a place in which to keep surplus funds. 

" botiaeholder .Bank as a convenience in paying bills - e'hrckmg aceuunt 

" thrifty person Bank at a source of interest on savings or small loans. 

“ property owner . . , Bank as a source of large loans on property. 

" investor Bank at a source of loans on collateral. 

" basinets man ... . Bank as an aid to busincu in short-time loans. 

“ coilege snidents of money and banking. 

" bank officials represent various degrees of additional knowledge, 

" egpert hankers of inrernational banks. 

“ economists specialising in money and banking. 
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more positive he is that he understands it fully. The expert who 
understands all the details listed above and others which might be 
added is probably much less certain that he understands banks and 
banking. The illustration affords a clear demonstration of degrees 
of understanding in the mastery of organized knowledge. 

Attainment of Precision in Knowledge of Functional Facts 

An analytical study of the trial results of the items in the 
WesleyTiold, Kelty, and Pressey tests by Dr. J. R. Strayer 
revealed another peculiarity of the learning process. The following 
examples will illu.strate this trait. 

A budget is a 

1. financial plan 2. bank statement 3. heavy club 
4. cooperative society’ 5. monthly payment. 

In the lower grades the pupils who had not attained the under- 
standing of the term required by the correct answer tended to 
select “monthly payment” as the answer. In the later grades “bank 
statement” was the preferred error. By the tenth grade practi- 
cally all of the pupils selected the correct answer. 

The same point appeared even more sharply in cases where 
these independent workers had used the same test items, though 
with different t)ptions. Note the difference in the performance at 
the same grade levels. 1 he.se percentages were obtained from the 
scores of several hundred pupils at each grade level. 

Example 1. 

Pressey What is meant by legislation? 

(a) the passing of laws; (b) the convicting of 
criminals; (c) the collecting of money; (d) the 
enforcing of laws. 

Kelty-Moorc Legislation means 

(a) men connected with the government; (b) 
state body for the stare only; (c) the making of 
laws; (d) the people w'ho make laws; (e) a con- 
gress composed of tw’o houses. 
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Scores : 

Grade 4 5 6 7 8 9 10 12 

Pressey 28 . . 41 72 79 87 

Kelty-Moore 15 24 15 18 26 23 

The Kelty-Moore item is apparently too hard even for the junior 
high school, the score of the ninth grade being practically pure 
chance. 

Example 2. 

Pressey If fifty people vote unanimously for a certain 

officer, how many votes did he get? 

(a) 25; (b) 49; (c) 0; (d) 50. 

Kelty-Moore Unanimous means 

(a) generous or unselfish; (h) the majority vote; 
(c) having the agreement of all; (d) the people 
who vote the same way; (e) almost everybody. 

Scores: 

Grade 4 5 6 7 8 9 10 12 

Pressey 75 82 85 . . 90 90 

Kelty-Moore 12 19 13 23 34 41 .. 

Here the Pressey item is so obvious that a pupil who had not 
noticed the term before might easily infer the correct answer from 
the question itself. 

Example 3. 

Pressey Who may be impeached? 

(a) a business man; (b) a criminal; Ic) a re- 
porter; (d) a president. 

Kelty-Moore Impeachment means 

(a) a letter that goes through the mail; (b) 
bringing to trial a high officer of the govern- 
ment; (c) challenging to a duel; (d) a fraud 
perpetrated upon an officer of the f^ederal gov- 
ernment; fe) finding that an officer is guilty. 

Scores: 

Grade 4 5 6 7 8 9 10 12 

Pressey 18 29 ^>0 87 96 

Kelty-Moore 13 15 10 18 46 38 
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In this case neither of the questions receives a score definitely 
above chance before the eighth grade. The Kelty-Moore question 
is, however, more searching. 

Example 4. 

Pressey Which is a mob? 

(a) a fleet of battleships; (b) a decrease in 
prices; (c) an excited group of people; (d) a tax 
on imports. 

Kelty-Moore A mob is 

(a) many things bunched together; (b) one 
who feels superior to other people; (c) any large 
group of people; (d) people who crowd around 
a speaker; (e) a disorderly or lawless crowd. 

Scores: 

Grade 4 5 6 7 8 9 10 12 

Pressey 66 . . 75 . . 92 . . 92 97 

Kelty-Moore 14 25 37 52 64 55 . . 

In this case the key word is not a “book" word but one which 
pupils have heard and seen often outside of school. Yet the low 
scores on the Kelty-Motire question indicate that the term is still 
imperfectly known at the end of the ninth grade. 

Example 5. 

Pressey \Vhich adjective refers to the country in which 

one is born? 

(a) alien; (b) foreign; (c) adopted; (d) native. 

Kelty-Moore A native is 

(a) Any person who lives in the country about 
which we are speaking; (b) a black person; 
(c) a person horn in the country about which 
we are speaking; (d) a wild person, not civi- 
lized; (e) a cannibal. 

Scores: 

Grade 4 5 6 7 8 9 10 12 

Pressey 43 65 81 90 96 

Kelty-Moore 11 23 36 65 67 57 

Phis is another instance of a word in general use outside as well 
as in the school. The difference in the scores, however, indicates 
that precision in its use is slow in developing. 
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Example 6. 
Pressey 


Wesley 


Scores: 

Grade 

Pressey 

Wesley 


When is an injunction most often used? 

(a) during a strike; (b) during a battle; (c) 
during an exploration; (d) during a voyage. 

An injunction is a 

1. part of speech; 2. wreck; 3. union of two 
things; 4. court order; 5. form of advice. 

4 6 7 8 9 10 11 12 

7 19 42 . . 55 69 

.... 5 12 37 46 59 K8 


Neither of these questions demands a very accurate knowledge 
of the term. The difference in the scores is intertsring. 


Example 7. 
Pressey 


Wesley 


Scores: 

Grade 

Pressey 

Wesley 


Which w'ord means freedom from slavery? 

(a) pardon; (b) agitation; (c) emancipation; 
(d) exemption. 

Emancipate means to 

1. enslave; 2. waste away; 3. liberalize; 4. make 
strong; 5. liberate. 

4 6 7 8 9 10 11 12 

22 25 59 69 86 

.. 25 26 48 67 62 74 


These two questions seem almost equally difficult. 4'he added 
distinction between liberate and liberalize in the Wesley test 
accounts for the lower score on this latter. 

In nearly every one of these examples, one ver.sion calls for a 
more precise understanding of the idea involved than doe.s the 
other. In the first version only one or two of the options have any 
real connection with the question and the choice is more easily 
made. In the second version more of the options relate to the ques- 
tion, the choice is more difficult and therefore calls for more preci.se 
knowledge of the idea involved. In examples 2 and 4 the simple, 
incomplete, and superficial knowledge of the idea demanded by the 
first version is met by the majority of pupils as early as the fourth 
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grade. The greater precision demanded by the second version is 
not reached by the majority of the pupils before the seventh grade 
in example 4 and is not yet reached at the ninth grade in example 2. 
The difference is not quite so great in the other examples but is 
clearly marked. The following question affords an illustration of a 
degree of understanding beyond the reach of most twelfth grade 
pupils. 

The people of a Congressional district are called the Congress- 
man’s (1) supporters; (2) opponents; (3) clients; (4) constituents; 
(5) henchmen. 

These illustrations would suggest that the mastery of single ideas 
is, like that of highly organized ideas, a matter of progressive devel- 
opment not attained at any one grade level hut built up in the 
course of a development extending from the fourth grade or earlier 
to the twelfth grade and later.‘ 

Where Pupils Acquire Functional Facts 

The cumulative learning of single functional facts revealed by 
these examples deserves further consideration. Fhe terms used in 
the Wesley-Gold, Kelty, and Pressey' tests are virtually all in cur- 
rent use, appear in daily newspapers or current literature, are, in 
fact, part of the ordinary business of life. Or. to stare this obser- 
vation in another way, pupils who read current literature are ex- 
posed to most of these terms constantly. 1 he selection of the items 
for the final test was accomplished by statistical procedure wholly 
unrelated to any specific subject matter offered in the schools. 
Items were retained or discarded on the basis both of progres- 
sive improvement in scores from the fourth grade to the twelfth 
and of the reliability coefficient of items within single grades. If 
the item showed steady improvement through all these grades, ad- 
vancing sufficiently above mere chance, it became a preferred item. 
If it behaved erratically, it was discarded, even though the erratic 
behavior might he most significant of the eft'ects of certain courses. 

*Thc phenomenon of progressive improvement in learning social material has 
also been noticed by Andcrsim and f.iiuitiuist of the University of Iowa. Cf. 
Anderson^ H, R., and Lindmiist, E. F.: Tkr Imprcrvemfni of Objeetiv* Tfstinf, in 
lltrtory. Second VcarbvMik of National Council for Social Studies. Pp. lOD-104. 

*Scc Chap. III. 
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The selection of items for the final form of the tests on the basis 
of continuous improvement without regard to the particular social 
science course offered at any grade level or, indeed, whether any 
course was offered, permitted the performance on the test to re- 
flect social learning gained through other means as well as through 
courses in social science. This fact was fully confirmed by the re- 
sults in the University of Minnesota High School, where the test 
was applied to all pupils in each of the six grades. It was con- 
firmed also in other trials. There is nothing startling in this con- 
firmation, how’ever. It only serves to emphasize an important con- 
dition of social science instruction, namely, that such instruction 
is not the only avenue to knowledge of society. 

The sources of social education are varied. 'I'hey may he divided 
roughly info three classes, (1) personal experiential (2) formal class 
instruction, and (3) extra-experiential information. The first con- 
sists of the pupil’s personal experience in his social life with his 
fellow's and the more or less directed experience he receives at 
home, at church, and in various organizations and in.stirution.s to 
which he or his family are related. I'his type of material i.s most 
vivid. The pupil recognizes himself as part of it. It impinges upon 
his several senses and usually involves his active participation as 
well. These psychological advantages render this potentially the 
most important source of his social learning. Unfortunately the 
lessons from this learning are ordinarily expressed in local idiom.s, 
not in the universal language of literate society. 

The third source is even more varied than the first. It consists 
of all those agencies for the transmission of remote, external infor- 
mation, the experience of others. It includes the pupil’s general 
reading in books, periodicals, and daily new'spapers; what he hears 
about the outside world, whether in conversation, from lectures, or 
over the radio; and what he sees as a detached oh.server either in 
passing, in pictures, or at the moving picture theater. Its relation 
to himself is not apparent; the experiences recorded are those of 
others, not hi.s own. I'he impression made upon him i.s much less 
vivid, reinforced hy fewer impacts upon his senses, involving fewer 
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of his feelings, and is usually fleeting, not persistent. The eflFect 
upon him will be largely dependent upon his own stores of apper- 
ceptive learning and interest. The third source springs from the 
ceaseless flow of apparently unrelated activities and ideas of the 
outside world. Its language is, however, of a more universal nature. 

Formal instruction in the .schools stands between these two. The 
material with which it deals is, like the latter, concerned with the 
thought.s and acts of the outside world. Unlike the latter, how- 
ever, it is organized. I he relationships have been worked out and 
the material is before the pupil in a permanent form to which 
instruction gives persistent force. I he experiences with which for- 
mal instruction deals have many points of similarity and often 
relationships with the more ephemeral material found in the out- 
side world. I hey also haye many points in common with the more 
direct personal c.vperience of the first source. 1 he skillful teacher 
can gain greater vi^ idness by pointing out these similarities and 
connections. 

One of the chief functions of formal classroom instruction, 
therefore, is to serve as the link between the other two sources of 
in.struction. The more efficiently this function is performed, the 
more effective will become the educative influences of the other 
two. Such instruction should enable the pupil to apply to the 
lessons of his own experience the universal terminology which the 
wider, grow n-up world employs, thus enabling him to identify him- 
self and his experiences with those of others in other parts of the 
world. It should also, by its careful tracing of relationship in wide 
human experience, enable him to recognize relation.ships in the 
ceasele.ss flow of otherw ise apparently unrelated incidents of the 
contemporary world. It should bring him nearer to the goal of 
recognizing in innumerable personal experiences functional facts 
bearing upon the more remote operation of the larger social world; 
and to a similar ability in tracing the bearing of larger ideas of 
organized society upon the more minute affairs of his owm daily 
life. 

I he relationship of learning in the schcwls to that from outside 
sources may, therefore, be likened to a chemical agent. It is the 
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means by which the other two are brought together. The two out- 
side educative elements, the experiential and the extra-experi- 
ential, are operating always. They have begun before school days 
and they continue through life. Most of the material of extra- 
experiential learning is of the inert variety. The functional facts 
inherent in it are easily overlooked. This can be corrected in large 
measure by the formal school instruction which, according to its 
quality, more or less accelerates the instructional value of the other 
two elements as well. The evidence of these tests indicates that 
this acceleration occurs during school days and continues when 
the formal instruction has ceased. 

Relative Proportions of Inert and Functional Facts 
IN the Several Social Studies 

After the first list of functional’ facts, drawn from the text and 
reference books in social science up to the twelfth grade, had been 
compiled, an attempt was made to classify the terms. Fhe task 
proved difficult because many terms belonged in at least two, some 
in three, categories. The more or less arbitrary classification 
showed approximately 45 per cent of the terms in political science, 
33 per cent in economics, 20 per cent in sociology, with less than 
2 per cent of ideas of time and space peculiar to history and geog- 
raphy. The more comprehensive list drawn from the junior col- 
lege work in social science and from newspapers showed a slight 
shifting, sociological and economic terms increasing. A study of 
functional facts found in selected textbooks for grade 7-11 inclu- 
sive showed sociological terms 49 per cent, political science terms 
42 per cent, economic 32 per cent, with the peculiar time and 
space ideas of history and geography together less than 1 per cent. 
The unusual total of percentage is due to the fact that over 20 
per cent of the terms were classified in two, some in three, cate- 
gories. These figures are of special significance in that, of the 
twelve textbooks analyzed, ten were textbooks in history, one in 
geography, and one in so-called unified social science. Nearly all 

*"Functionar' facts as here used arc limited to the restricted list as described 
pp. 8*15. 
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the functional facts in history and a large percentage of those in 
geography are functional facts of social science. Those who ana- 
lyzed the several types of material for the comprehensive list of 
functional facts found the books in history and geography high , 
in “inert” facts and relatively low in “functional” facts, while the 
books in social science were very high in “functional” facts with 
a minimum- —in some cases almost none — of “inert” material. 

Implications of Studies in Understanding 

The Probable Process of Learning, Concepts, Ideas, or Functional 
Facts in the Social Sciences ^ The import of these observations would 
indicate that the learning process in the social science field could 
be illustrarcd somewhat in the following manner, the concept 
“stare” as a governmental division of the United States serving 
to describe the general process. 

1. The child has heard the name of his own state and of other 
states fretjuently before he ever goes to school. 

2. Most children will have asVed what a state is and may have 
received some form of dictionary definition, usually no longer than 
a sentence or two, which they grasp vaguely. 

3. In the stories of the history of the country, incidents, events, 
and persons will be connected with certain states. 

4. In geography, the distribution of resources, population, cli- 
matic conditions, occupations, and living conditions will usually 
be referred to in terms of states. 

5. Any systematic history of the United States will cover the 
settlement of the states, the origin of their settlers, organization of 
their governments, the relation of the state to the national govern- 
ment, various incidents in each state’s history, and the develop- 
ment of peculiar traditions and attitudes. 

6. In the course in civics, the child will learn some details about 
the way in which his state government is organized and how' it 
operates. 

7. Meanwhile he will have noticed allusions to various states 


'See also pp. 14-16, 113-1 
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in the newspapers, periodicals, and other news-disseminating 
agencies. 

8. He will not have attained a working understanding of “state” 
• as a governmental unit of the United States until he has learned 

wherein New' York, and Nevada, Texas, and Rhode Island, Massa- 
chusetts, and Mississippi qualify as such states, while the District 
of Columbia and Alaska do not. 

9. Most pupils will have attained this degree of understanding 
at the end of their high school course. If they continue in college 
past the sophomore year, some of them will have had the oppor- 
tunity to take systematic courses in the history and government 
of their own particular states. 

10. 7'he average citizen charged with some responsibility for the 
conduct of affairs will continue to enlarge his knowledge and 
understanding of his own state and others, though not in a sys- 
tematic way. If indifferent to politics, he will learn little, if any- 
thing, perhaps not enough to keep alive the functional facts that 
he learned in school. If professionally Interested in politics, a 
politician, he will learn much about his own state and even others. 

11. A few' w'ho are fascinated by the study of local politics or 
political science will devote their lives to a study of state govern- 
ment and w'ill still have something further to learn about the 
subject when they cease working. 

This illustration may be applied equally well to larger or smaller 
concepts. It can be applied to international finance, international 
politics, or even to so limited a problem as the relations between 
tw'o individuals. In the case of the larger concepts, the working 
understanding will be attained later and by fewer individuals; in 
the case of less difficult concepts, earlier and peril aps by more 
individuals. In any case absolute, final, and definitive knowledge 
w^ill never be attained, whether of international finance or the 
continuing relationship between two people. 

Peculiar Nature of Ideas in Social Science. I hc illustration of 
the probable process of mastering an idea in social science suggests 
that there is a marked difference between tlie ideas taught coin- 
nionly in the social sciences and those taught commonly in most 
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sciences, technology, mathematics, Jogic, and philosophy. Ideas in 
social science are not definite in the sense in which ideas may be 
definite in some of the sciences or technology, for example. The 
physical sciences may present their ideas in precise formulae, and 
the formulae can be applied with equal precision, insofar as the 
scientist or technologist controls his material. In the chemical and 
physical laboratories, in the industries wdilch fabricate raw ma- 
terials, the raw materials can he refined or shaped in such a way 
that the formulae may be applied with almost absolute precision. 

I be raw material of the social scientist is life itself over which he 
has very limited, if any, control. Most of his ideas are derived 
from the observation of society, and if he indulges in logical specu- 
lation for the purpose of explaining, improving, or controlling 
society, he still has the task of modifying liis theory or formula to 
hr the actualities of social life. His theory or formula is valid 
only to the extent that it conforms to the ways of social life itself, 
or that social life can he modified. 

Those ideas in social science which are derived from the ohser- 
vation of social relationships all represent a range of variation. 
They are derived from a series of recurrent, similar incidents, no 
two of which are exactly alike. Tlu-ir statement in brief verbal 
furnulae .such as ordinary dictionaries emplo>’ serves to designate, 
but not to define. If the statement succeeds in characterizing the 
simple common denominator of the range of incidents involved, it 
may not cover a single actual occurrence of the idea. Equipped 
with a dictionary definition alone, the pujiil might not be able to 
identify a single instance of the idea. He is like the finder of a ring 
of keys. To the finder they are only jingling metal like a dictionary 
definition. For the man who really owns them they will unlock 
many things. It is likewise with a social idea. A firm grasp of the 
idea i.s possible only if the pupil learns the typical range and the 
limits of the idea. He may begin his learning of the idea either by 
meeting it in his experience, whether direct or through reading, or 
by a verbal statement, a so-called definition. He does not possess 
an inclusive-exclusive knowledge of it, however, such as the scien- 
tist demands of a definition, until he has met the range of its 
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application, including the limits before which and beyond which 
it acquires a different meaning designated by a different term. 
The truth of this statement is demonstrated by the progressive 
improvement in the scores on the Wesley-Gold, Kelty, and 
Pressey tests. 

Those ideas in social science which are derived from logical 
speculation about society in an effort to interpret, control, or 
improve society may be learned as verbal formulae to satisfy 
purely academic or intellectual demands. 'I’o become a part of 
social science, it is essential that they be derived from an accurate 
and comprehensive knowledge of society, and that they be effec- 
tively applicable to society. The task of the social scientist, then, 
becomes one of determining how much modification his logical 
formulae must undergo in order to he applicable to society. The 
illustration afforded by Montesquieu's speculatirui about the suc- 
cess of the English Government is pertinent. His theory that it 
lay in a rigid separation of powers, legislative, executive, and 
judicial, was expounded just in time to be taken over into the 
Constitution of the United States. "I he disparity between the fact 
that in actual practice they cannot be rigidly separated and the 
persistent effort to keep them separate has been a source of endless 
confusion not yet corrected after a hundred and fifty \ears. To 
meet the needs of social scjjence understanding of such ideas 
would require a modification or qualification of the logical formulae 
to meet the needs of most, if not absolutely all, social relationships 
to which they apply. 

Learning in the social sciences, therefore, is a constantly cumu- 
lative process which is only well begun by the time the piqul 
reaches the end of the public school system. Even the most sim[>le 
of its ideas cannot be firmly grasped as a result of one incident or 
one statement of principle. Few of them are mastered even to the 
extent of a good working knowledge in any one grade. But their 
learning is begun as early as life itself and may be greatly acceler- 
ated by wisely directed instruction during the school years. 'I'his 
direction would seem to consist primarily of a judicious mixture of 
inert examples and functional facts, each teacher in the process 
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aware of the examples already met, whether in previous instruc- 
tion or in the business of life, and ready to widen the pupil’s under- 
standing either by fixing more firmly the functional facts already 
grasped or by initiating the understanding of additional functional 
facts. 

Interrelationship of the Social Sciences. The social sciences have 
been separated into various departments for purposes of advanced 
study. The separation is necessarily artificial. No actual social 
event, no social organization or institution, can be classified accu- 
rately as exclusively economic or political or social. The larger or 
more complex it is. the more certain it is to have aspects of all three, 
and even more, categories. I he difficulty which the investigators 
finind in classifying even many of the simple functional facts in a 
single category is suggestive. A tariff is both economic and politi- 
cal and is certain to be social m its consequences as well. An in- 
dustry, especially iflarge, is certain to combine all of these elements 
as well as many others. I he upper officials will have most of their 
energies involved in problems of organization, formulation of poli- 
cies and regulations, administrations, maintenance of morale, rela- 
tions with other industries, with governments, and with the public 
at large, all of which are ordinarily classified as political functions, 
fhe lower officials and workers will have chiefly technical prob- 
lems involved in the industry, though not without some political 
and some social concerns as well. The marketing of wares involves 
a consideration of social needs, desires, practices, as well as the 
more purely economic factors of transportation and storage. The 
same complex of factors is found in any organization or institution, 
whether its dominant function he social, economic, or political. It 
is equally true of stare and church, industry and education, com- 
merce and recreation. None of them can be understood fully in 
terms of one social science alone. 

I hough social learning is derived from many sources, most of 
them outside of school, the service of the social science subjects in 
the schools is one of economy in that learning. The economy in 
social learning accomplished in the schools is twofold. Finst, it may 
hasten the pupil’s grasp of functional facts essential to his dealings 
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with the experiences of social life by supplying him with the typical 
range of illustrations involved in the functional facts. This should 
be of great help to him in dealing wdth the incidents of social life 
W'hich he does not control. Second, it may widen the pupil’s appre- 
ciation of the relationships in human affairs so that he will recog- 
nize more fully the extent of those affairs to which any given for- 
mula of interpretation, improvement, or control will apply and by 
which it will be modified. 

The importance of the second of these services is not commonly 
realized. I he universal desire to arrive at conclusions as quickly 
as possible interposes several important obstacles to such appre- 
ciations. One of these is the preference for sharpl\' defined informa- 
tion. Whereas the social scientist, as the result of bis long study, 
comes to the realization that specific events and specific institu- 
tions are merely devices by which society is working out its prob- 
lems, that the underlying process is the important lesson, the 
beginner prefers the definite events and specific institutions. 1 his 
obstacle is doubly exasperating because the beginner can only 
learn to appreciate the importance of process by learning a series 
of specific changes in society's solution of the problem, and be- 
cause the social scientist has usually forgotten that it was in this 
way that he himself learned to appreciate the underlying process. 

Another obstacle arises from the univer.sa] fascination with the 
possibility of a single explanation of human affairs. 1 hi.s urge for 
a philosopher’s stone in the operation of society has in the past 
led to a variety of theories; a religious interpretation of history, 
illustrated by St, Augustine; the determining iiiHiivnce of pinsical 
environment as elaborated by Buckle; the economic interpretation 
of society in various forms from Malthus, Adam Smith, and Karl 
Marx to the many modern devotees; the political interpretation .so 
eloquently expounded by Freeman and Seeley; and, most recently, 
the psycho-analytical interpretation derived from Freud hy many 
contemporaries. It is only with the utmost reinctance that the 
social scientist himself has come to recognize that there is no one 
explanation of human affairs, that all of the interpretarion.s which 
have been so enthusiastically advanced are but factors in the in- 
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tcrpretarion of large social phenomena, and that, in addition, there 
are probably many other factors which remain still to be identi- 
fied. I he beginner’s weakness for the single explanation ma^' there- 
fore be condoned, but it is desirable from the point of view of the 
public welfare to wean him from that preference as soon as possible. 

One goal of instruction in the social sciences is to extend the 
power of each individual to recognize the range of individuals and 
institutions which will l)e involved in any given program of reform 
or control, or even in any social activity of that individual himself. 
Nothing in past human experience justifies the hope that this goal 
will ever be fully attained. But human experience abundantly 
jtroves that .society gains directly in proportion to its progress to- 
ward that goal, and no society has had a greater opportunity to 
approach that goal more closely than is afforded by the condition 
of practically universal high school education which now exists in 
this country. 

One of the most imjiortant outcomes of learning in the social 
studies is the appreciation of the underlying process which parti- 
cular institutions, specific measures, and definite events serve. It 
is ai.so a fond hope of most social scientists that younger students 
may be led more rajudly, if not immediately, to the appreciation 
of the process instead of being satisfied with a knowledge of events, 
measurc.s, and institutions. Ibis goal, however, is only attained 
w'ith very great difficulty. 

I bis difficulty, or these difficulties, may be illustrated by the 
example of the railroads. For society the railroad has been prob- 
ably the most important agency in the formation and maintenance 
of modern urbanized and industrialized civilization. I he economist 
has recognized it as the chief device of the process of transportation 
in modern times. Most railroad workers, and many railroad offi- 
cials, including even executives*, hoards of directors, and especially 
governmental regulating bodies, have been distracted by the im- 
mediate material elements of rails and cars, office buildings, and 
organization, to think of rlu ir work as railroading, pure and simple. 
Many of them may not have carried their thinking far enough in 
the first place to appreciate the importance of the process, while 
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Others may have forgotten. The result of this limited view, when 
automobiles and, later, airplanes became efficient agencies of 
transportation, threatened to be very disastrous. Instead of wel- 
coming these developments as aids in Increasing the efficiency of 
transportation, railroad men frequently took a hostile attitude and 
resented them as interfering with railroading. It would be difficult 
to estimate the wastage in money alone caused by the struggle of 
railroad men to restrain the operation of auto trucks. Years passed 
before the more thoughtful railroad operators realized that their 
business was to aid in the process of transportation, and that this 
process could be greatly facilitated b}' linking both automobile and 
airplane to the railroad in such a way as to combine the pecuhar 
advantages of each in the interest of society as a whole. (Govern- 
mental regulating authorities have been slower still to recognize 
the situation. If this e.xample illustrates the wastefulness of mis- 
taking the institution for the process and the superior importance 
of realizing the underlying process, it equally well illustrates the 
difficulty of gaining and maintaining the idea of process. 

Every field of social activity, however, affords innumerable illus- 
trations of the difficulties that lie in the way of the attainment of 
the idea of process. The constable who, when electric lights were 
first installed in automobiles, insisted that drivers go out to the 
front of the car to turn on the lights furnishes anotlier illustration. 
The old-fashioned judge w'ho was so meticulous about the letter 
of the law and decided cases on that basis, rather than on the 
basis of the social process being served by the law, supplies another 
example. The school teacher who is intent upon the exact form of 
a new method and disregards the process of education w hich that 
method is intended to serve affords yet another illustration. I he 
sulmrdinate in any organization, economic, political, or social, who 
discharges his duties with a painful regard merely to the letter of 
his orders encounters the same difficulty in appreciating proce.ss. 

These difficulties, so common in every field of .social activity, 
suggest that the task of appreciating process is it.self difficult. I hc 
attainment of the idea of process would seem to he achieved only 
through a thoughtful acquaintance with a vertical series of events, 
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measures, and institutions. The material evidences of the events, 
measures, and institutions are grasped first — places, persons, and 
buildings. The immediate purposes which these serve are grasped 
next. The underlying continuous social process being served can 
only be grasped after a series of such material and immediate 
situations serving the same society at successive intervals are com- 
prehended. I'he comprehension of process is therefore a feat of 
sustained thought, a complex abstraction, as much more difficult 
to reach as it is more abstract and complex than a simple func- 
tional fact. Every normal individual may be expected to attain 
to a limited t om prehension of process in that field in which his 
thoughtful work lie.s. Hut the attainment of a wider comprehen- 
sion of the numerotis processes underlying the activities and in- 
stitutions of a complex society can only he achieved by a very 
superior scholar in tlie field of the social sciences. 

n. HI.SlOkK'AI.-MINDhn.NKSS AM) CTLirKt; PAl TERNS 

I lie list of values for wliich tests were projected included the 
term ‘’historical-mindedncss.’' This inclusion was indicated as de- 
sirable by the advisory committee of historians; and workers, both 
technical and subject-matter experts, were found to undertake the 
construction of a test. Historical-mindedness was defined as the 
habit of judging events and practices in light of the times in which 
they occurred. I'luis, for example, the statement that a president 
of the United Stares ke|>t slaves would he received in very different 
fashion in the twentieth century from its reception in the eigh- 
teenth century. Further analysis of the problem indicated that 
the attainment of this value definitely involved appreciation of the 
culture pattern of the time. Historical-mindedness differed from 
culture pattern as that term is used by the sociologist or cultural 
anthropologist only in the addition of the time element. The effort 
to test the student’s possession of this value involved two steps, 
(1) knowledge of culture patterns, and (2) appreciation of the fact 
that culture patterns change in time. Fhe second of these de- 
pended upon the fir.st. Such a test, if constructed, would also have 
value for the sociologist who is concerned with differences in cul- 
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ture patterns according to place. The definition of the culture pat- 
tern would involve both time and place and the historian, too, 
would be concerned with difFerence.s according to place at any 
given point of time. The test of culture patterns promised to be 
of some service also to the economist and political scientist. I'hus 
this value, which at first seemed peculiar to one field of social 
science, promised to be of general concern to all of them. Further- 
more, the geographer’s concept of “regional personality’’ is closely 
related to it. 

Analysis of PaUerns. The effort to analyze patterns of culture 
proved to be more arduous than was anticipated. It v,as found 
difficult to list the distinctive characteristics of a single period of 
history. Certain elements, such as Inventions and discoveries, 
could be sharply differentiated as specific achievements. I heir 
effect upon the life of the times, however, was a matter of gradual 
development and diffusion. It was not easy to determine how soon 
after the invention of gunpowder castles cea.sed to be efiicrive 
fortresses and became merely residences, or how soon after the 
discovery of America this knowledge of new opportunities entered 
actively into the thinking of a large part of Kuro[)e. It was a|>par- 
ent that these effects would be felt differently in different portions 
of Europe. The historical pattern was thus affected by differences 
in region as well as time. Once this fact was recognized, it raised 
a corollary question whether distinctive patterns of one place at 
one time might not occur in practically the same or similar 
form at a very different time in another place, d he manorial sys- 
tem of agriculture is generally regarded as characteristic of F.urope 
in the Middle Ages, yet nearly every element which would he 
included in the pattern of such manorial life would appear in the 
eighteenth or even nineteenth century in eastern Prussia and per- 
haps elsewhere in Europe. Further study and reflection indicated 
that nearly every pattern of social life found in time might also 
be found today in some spot, the difference being largely in the 
extent to which that pattern applied, a pattern common to most 
of society at one time still existing only in isolated communities 
today. This would suggest that a rest of historical-mindedness 
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must he limited to some dcHmtc area and must include changes 
in the pattern from time to time in that area. 

A consideration of patterns in limited areas revealed further 
difficulties. The same pattern did not hold for all people in the 
area, d’he pattern varied with the social classes, the upper classes 
being in touch with a much wider social world, while the lower 
classes lived in a very restricted pattern. The difference between 
these patterns was not sharply defined, for the classes fused into 
one another by almost imperceptible degrees until it seemed desir- 
able to discover the pattern for each individual. Such degree of 
detail, however, was beyond the reach of any student within the 
years covered by this Investigation. No given area, even in Amer- 
ican history, could have been studied intensively enough to afford 
the opportunity to test such knowledge. 

Incidentally, the possibility of limiting the pattern to some 
phase of life or living was also canvassed. Institutional patterns 
such as forms of government, religious creetis, and economic sys- 
tems were considered. Many of these had definite patterns, as, 
for instance, constitutions of states, or church creeds, which are 
set down in written form. Here, however, there was found to be a 
great diftcrence between practice and theory. 1 he pattern of a 
church creed and dogma as set forth in documents described the 
actual pattern of belief held by relatively few of the individuals 
who professed it. A given church might include within its mem- 
bership some whose pattern of religious observance was almost 
fully descrilied by the w ritten dogma of that church. On the other 
hand, that pattern might definitely vary among the individual 
members of the church all the way from that point to the indi- 
vidual who.se only point of contact w ith it was a nominal member- 
ship in the irrstitution. I he same range was found in the case of 
political governments and parties and economic systems. It seemed 
necessary, therefore, to conclude that the patterns described in 
such sy.stems could not be used effectively in attempting to test 
historical-mindedness. 

Anthropology has described culture patterns of primitive socie- 
ties and recently geographers have developed the concept of 
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“regional personality” which is a pattern of physiographic and 
human activity elements. Both seem to offer an opportunity to 
determine such patterns with a completeness and precision re- 
quired for purposes of test construction. The “regional person- 
ality” involving the relatively permanent geographical elements 
promised to be of real help toward the development of patterns 
required by historical-mindedness. Those regions which had been 
well populated over a long period of time and which had witnes.sed 
extended historical development, however, showed changes in the 
pattern of activities from time to time. It appeared, on inspection, 
that the more prominent in history a given region, the more likely 
it was to show such changes in activities and, consequently, in 
culture pattern. The less populous, less important in history, a 
given region, the more permanent the regional personality activity 
and culture pattern. This observation proved a further disappoint- 
ment in the effort to test historical-mindedness. 

Parker and Calkins, in their plans for a series of tests of geo- 
graphical understanding, had included a test of regional person- 
sonality. This proved, however, to be one of the most difficult of 
the tests in their series. Part of the difficulty lay in the definition 
of the complex factors involved in the several regional personalities. 
Perhaps even greater difficulty w^as found in formulating questions 
which would test this knowledge effectively. As a result, the test 
could not be completed in time to serve the needs of this Investi- 
gation. Such tests are contemplated in the program of the Na- 
tional Council for Geography and w'ill doubtless appear in the near 
future. Meanwhile experience in testing the understanding of 
single ideas had involved so much effort, both in the formulation 
of the tests and in the statistical analysis of trial results, that the 
effort to test the more complex ideas involved in the culture 
patterns of historical-mindedness had to be abandoned. 

Our limited experience might suggest that patterns, whether of 
action, thought, or circumstance, or the combination of all three in 
culture patterns, are chiefly theoretical. Whether laid down as 
programs or derived from observation, they do not exactly de- 
scribe any individual or even group concerned in them. 'I'hough 
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this may be true ot the complete pattern, it does not impair the 
value of the pattern as a guide to the understanding of its imper- 
fect reflection by individual or group. It merely adds to the diffi- 
culty of using the pattern, calling for constant alertness in appli- 
cation and a readiness to accept incompleteness in detail as cir- 
cumstances and conditions require. Awareness of such patterns as 
regional personalities, historical-mindedness, and other general 
culture patterns can be taught fairly early; proficiency in the ap- 
plication of these patterns can not be expected until late in the 
career of the student. He may select the most concrete and obvious 
elements in the pattern at first and graduall}' enlarge his use of 
additional elements as his knowledge increases. 

Ill, SKILLS 

.Another class of values recognized as common to all the social 
science subjects was designated under the caption “skills.” The 
appropriateness of this designation may be questioned, since the 
term is commonly associated with manual and mechanical pro- 
cesses. 1 he tools and instruments employed in the social science 
subjects require mental rather than manual dexterity. Whether 
the applications ol the term “skills” is a legitimate figure of speech 
in this connection is, therefore, doubtful. Perhaps “study activi- 
ties” or “habits” may represent a more accurate choice of words, 
though neither is (juite adequate. At any rate, this section deals 
with those methods of finding and treating information which are 
supposedly acquired in the course of these studies and are expected 
to be serviceable to the individual in dealing with similar infor- 
mation in later life, d he acquisition of these methods of study may 
properly be likened to acquaintance with, and practice in, the use 
of tools in any more mechanical type of activity. 

It may be questioned whether such skills should he considered 
apart from the materials with which they deal. Few pupils could 
he won to acquire skills as skills, and few teachers would feel justi- 
fied in teaching them as such, ^’et teachers realize that these skills 
are learned in connection with materials that are easily forgotten, 
and they expect the students to retain the skills for application to 
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new materials that are similar. A student is expected, for example, 
to know W'here to find information on technocracy and how to 
evaluate the information, even though the particular subject was 
never studied in class. 1 hus, from the instructional standpoint, 
skills have a large independent value and constitute one important 
class of outcomes of instruction in this field. 

As in the case of understanding, the work of constructing tests 
to determine the pupil’s mastery of these skills entailed much pre- 
liminary work. The first task was to identify the various skills 
required by the social science subjects. Secondly, it seemed desir- 
able to analyze these into their component details which might be 
made the basis of test questions. Incidentally, there was ever 
present the question whether a given skill was peculiar to these 
subjects. A third task was to discover for which of these skills 
satisfactory tests were already available. I he fourth step wa.s to 
decide upon the most promising prospects for test construction. 

The first task, that of designating the specific skills involved in 
the social science subjects was greatly facilitated by the work of 
Dr. H. E. Wilson, of Harvard University, who had long been inter- 
ested in the problem of study activities. As a further aid to the 
Investigation, one of his students, K. A. Price, conducted a ques- 
tionnaire inquiry of teachers throughout the country. Members 
of the staff of the Investigation and teachers of the Minnesota 
Seminar also devoted some attention to this problem. 

The composite findings of these teachers and investigators may 
be generally classified in three broad divi.sions: (1) those concerned 
with the acquisition of information; (2) those concerned with the 
evaluation of information, and (3) those concerned with the expres- 
sion of social learning. 

The Acquisition <?/ Information, From the simple ac(}ui.sirion of 
social information through sight and sound, the first type of .skills 
moves on to include acquaintance with the devices and accus- 
tomed channels for the collection and distribution of social infor- 
mation. This involves such matters as tables of content and irt- 
dtees of hooks; cataloguing devices of libraries, such as card index 
and classification systems; standard reference works, encyclope- 
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(lias, atlases, fjazetteers, biographical dictionaries, or yet more spe- 
cialized works, like the Dictionary of American Biography and the 
Encyclopedia of the Social Sciences; guides to periodical literature, 
including newspapers and more specialized references such as the 
^ earhooks, or the United States Census; hihliograpbical guides on 
tields ol history, politics, economics, and sociology. 

Ability to read might he considered as the fundamental skill of 
the social studies, hut as this skill is understood by teachers in the 
field it is highly complex. The phy.sical and psychological aspects 
of the skill arc ordinarily not the concern of these teachers. Fortu- 
nately, in recent years psychologists have been devoting much 
profitable labor to a study of the entire reading process. The 
teacher of the social science subjects is. however, more concerned 
with Its later stages. These involve knowledge of vocabulary, ac- 
c|uaintance with the symbols devised to summarize large bodies of 
information, and familiarity with the ideas that recur frequently. 
Ability to read efficiently is contingent upon such knowledge, and 
speed in reading, which at first glance seems merely a mechanical 
process, proves, upon analysis, to be a compound of apperceptive 
knowledge and ability to control eye-movement, the former being 
far the more important element. In his study of skills or study 
habits. Price found teachers listing “reading” as most important 
through the twelfth grade. It is the apperceptive knowledge essen- 
tial to reading, rather than the mechanical process involved, which 
causes most of the difiiculty in^thc later grades. The students find 
themselves halted hv terms, expressions, symbols, and ideas which 
they do not understand. In addition, students have to learn to 
discriminate, as Henry Johnson pointed out in his Teaching of 
History, between different kinds of reading. The purpose and man- 
ner of the re.ading matter, as well as the purpose of the student in 
reading this material, are important conditioning factors in the 
•skill of reading. 

Ability to read i.s aided by yet another set of skills, namely, the 
practice of analysis and note-taking. The ‘implc outline of para- 
graphs, the more detailed notes, the more clasely organized outline 

the larger reading units contained in entire chapters and books. 
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practice in filing notes from several sources dealing with the same 
topic, and practice in classifying and filing information from varied 
sources, all represent stages in the development of analytical skill. 
Teachers have devised aids in the development of analytical skill, 
such as outlines and syllabi, and guidance or study outlines. The 
ability to analyze, like the ability to read, is a complex skill depen- 
dent more upon the student's previous knowledge than upon me- 
chanical arrangements. Like reading, its attainment .seems to re- 
quire the constant attention of the teacher throughout the second- 
ary school years. 

The reading and making of maps is a fundamental skill in geog- 
raphy, an essential skill in history, and more or less necessary to 
all the social sciences. It differs from the reading and analytical 
skills just discussed in that it obv'iously involves a technical vocab- 
ulary consisting largely of established conventional symbols. After 
these are mastered, the problem has many points of similarity to 
other kinds of reading and analysis. Knowledge of the meaning of 
the globe, lines of latitude and longitude, land and water configu- 
ration, and an understanding of the symbols for mountains, streams, 
lakes, and desert, cities, and states, and climatic features represent 
only parts of the special vocabulary demanded. In modern times 
the social scientist has imitated the geographer in devising graphic 
symbols and statistical charts for data of varying degrees of com- 
plexity. This practice of graphic repre-sentation of highly complex 
know’ledge and massed data has led to much confusion. The use of 
pictorial devices commonly known has caused even children to a.s- 
sume that they understand the ideas involved because they recog- 
nize the pictures. The compression of large and complex bodies of 
knowledge into diagrams and charts which the eye can see at a 
glance has beguiled many wnth the illusion that they understand 
because they see. Such material, on the contrary, usually repre- 
sents highly concentrated information and generalization with wide 
ramifications of detail which only those who know the detail can 
grasp. The principle of William James about generalizations ap- 
plies here with special force. Ability to understand such charts and 
graphic symbols in social science is a matter of cumulative advance 
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in apperceptive knowledge, usually not achieved until the later 
years in college, and then in varying degree, according to the 
amount of knowledge at the student’s command. 

The acquisition of this vocabulary, so graphically expressed that 
the eye sees it at once, is also a slow process, far from complete at 
the conclusion of the secondary school. Apparently its mastery is 
also a matter of cumulative advance in apperceptive knowledge. 

Skill in aural acquisition of information, though much demanded, 
has received relatively little attention from teachers or psycholo- 
gists in recent years. The development of the radio and its in- 
creased use for the discussion of public affairs has increased the 
demand for this skill. It is one so constantly employed in instruc- 
tion, almost exclusively in pre-school years, that its acquisition is 
probably assumed throughout. 1 he wide condemnation of the lec- 
ture method by educational psychology some years ago has prob- 
ably deterred teachers from consideration of this skill. As a result 
it receives attention only in those colleges where the instructors 
still insist upon examining the lecture notes of their students. It 
may be seriousl> questioned whether such neglect of this skill is 
justihed.' 

The Ex'ulualion u/ / njormation. The skill most difficult to acquire 
in rhe Held of the social science subjects is that called critical 
ability. It is highly complex. It assumes proHciency in the other 
skills, has certain procedures of its own. and involves at every turn 
the application of judgment. It requires great refinement before it 
can be accepted as part of instruction, w hether in school or in life. 
1 he process of tbi.s refinenu nt consusts chiefly in the application of 
rules of evidence as those have been developed in centuries of 
study. 1 hose rules have been described for the treatment of oral 
testimony in law and written tcstimon> in history. As such they 
serve all social science, 'fhe treatment of them by Langlois and 
Seignohos is generally regarded as standard. Each new device or 
method of disseminating social information develops supplement- 
ary refinements in the process of dealing critically with its material. 

'(’f. di'.rmsiun of oial mstriictioii in Horn: Stutiy of Aftthotlt ttf InstrucliQn. 
Scribners, 195*1. 
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Usually these are met by a modification of the rules already known, 
though some of these devices, such as the modern newspaper, the 
cinema, and the radio, have raised so many additional critical prob- 
lems as to justify extended separate treatment. 

Certain elements of criticism, however, are fundamental. I'hese 
are of concern to both teacher and pupil. All social information is 
relative. The person reporting is quire as important as the infor- 
mation that he reports. Did he see everything? Was he qualified to 
appreciate or understand what he saw? Did he write as soon as he 
saw it or afterward — perhaps long afterward? Did he have any 
reason for reporting the event otherwise than he actually saw it, or 
for selecting only some particular aspect of it? In other words, the 
qualifications of the writer or reporter, the circumstances of the 
writing or reporting, and the possible bias of the reporter are ele- 
ments which enter into the appraisal of the material reported, no 
matter what the medium through which the report is made. Next 
to the author and his circumstances, is the question of probabilities 
of correctness or error. Given the surroundings and actors in an 
event at a given time and place, is it likely that the account is cor- 
rect? Does it conform in all details to the known details of the time 
and place? How much of the account may be accepted as certain, 
how much as conjectural? All of these are considerations which 
condition any account of social information. 

The development of the critical skill is a long and arduous pro- 
cess and is expected to continue after the student leaves college. 
The skilful teacher can make important use of the practical learning 
which the student is acquiring in appraising information that he 
hears from his fellows. Even young children will have learned to 
take into account the expression on the speaker’s face, his attitude, 
and the tone of his voice as well as some of the considerations al- 
ready mentioned. I'he application of this same critical judgment to 
written and printed material is, of course, more difficult, hut the 
same problems are involved. Comparison of new information with 
that already possessed is involved in both kinds of information, 
written and printed material, calling upon a yet greater body of 
learning. In both cases, knowledge of the person furnishing the 

[ 62 ] 



RANGE OF SKILLS 


information, of his qualifications and his purpose, is necessary, 
though the written and printed material presents greater difficul- 
ties. Accurate knowledge of place, time, and circumstances of the 
scene and setting of new information, so essential to the judgment 
of probability, is also common to both classes of knowledge, though 
here again the w’ritten and printed material dealing with events 
more distant in time or space, or both, adds to the difficulty. The 
difference is largely a matter of degree and further refinement as 
the student’s body of social information is enlarged, his ability to 
prov ide himself with the essential further information improved, 
and his alertness cultivated. It is a cumulative process taking on 
additional refinements of technique as the social learning widen.s 
and deepens from the horizon of the playground toward the world- 
wide interrelationships of economic, political, and social affairs. 

Expression of Social Learninj.]. While skill in obtaining informa- 
tion from oral sources has been neglected, even shunned, in recent 
years, there has been increasing attention given to activity of var- 
ious kinds. I bis might be described as the development of skill in 
e.xpre.ssion of social learning. I'he traditional forms of expression 
used in cla.ssrooms since there were classrooms, namely, oral reci- 
tation and written composition on social themes, have not received 
proportionately enlarged attention. In fact, the increased use of 
the new-ty[>c test has lessened somewhat the emphasis on those 
forms of expression. On the other hand, much attention has been 
devoted to the development of activities of great variety, practice 
in the use of parliamentary rules, practice in drawing, designing, 
reconstructing, collecting, dramatization, and re-enactment of 
scenes related to history and other social science subjects. All of 
these devices arc intended to afford opportunity for individual self- 
expression and, incidentally, to cultivate added skill in that form 
of expression. 

Range of Skills 

The following table affords some Impression of the range of 
skills recognized in the teaching of these subjects, as it also indi- 
cates the judgment of teachers regarding the relative importance 
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and usefulness of these skills. Questionnaires were distributed to 
approximately seventeen hundred teachers and nearly Hve hun- 
dred replies were received. This table is taken from a preliminary 
report based upon the tabulation of the results on one hundred 
returns, selected at random, for each of the six grades indicated. 
Though confined within these grades, the tables throw some light 
on the practice of teachers both before and after these grades. 

The following list presents the fifty-two items in the order of 
their appearance on the questionnaire. 

A. Visual Activities. 

1. Read to understand: Read to comprehend all of the ideas 
contained in the material. 

2. Read to memorize: Intensive reading and re-reading of 
material to fix associations and allow immediate recall 
when desired. 

3. Read to locate information: Relatively rapid skimming 
through a bmly of material for the purpose of locating and 
selecting specific data. Example: Look through the Con- 
stitution to find all the clauses that refer to the courts. 

4. Read for enjoyment: Extensive rather than intensive read- 
ing which appeals to the pupils’ emotions as well as to the 
intellect. Example: Reading historical novels and poems. 

5. Study maps to understand all the ideas they contain. 

6. Consult maps to locate specific information. 

7. Study charts and diagrams to understand all the ideas they 
contain. 

8. Consult charts and diagrams to locate specific items of 
information, 

9. Study graphs and statistical tables to understand their 
full significance. 

10. Consult graph.? and statistical tables to locate specific items 
of information. 

11. Observe pictures, scenes, models, relics, exhibits, bulletin 
boards, etc., for general impression and emotional enjoy- 
ment. 
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12. Observe pictures, scenes, models, relics, exhibits, bulletin 
hoards, etc., to locate specific items of information. 

13. Observe pictures, scenes, models, relics, exhibits, bulletin 
boards, etc., for specific details and emotional enjoyment. 

B. Listening Activities. 

14. L isten attentiveU'^ to oral presentation of materials in 
classroom: Class discussion, declamation, or dramatization. 

15. Listen to music in classroom: Listen to phonograph records 
or group singing. 

16. Listen to radio programs: Listen to radio music, addresses, 
dramatizations, current events, etc. An activity impersonal 
in character, and which may involve a somewhat different 
psychological situation. It may he carried on at home or 
in school. 

17. Listen to illustrated lectures: An activity involving both 
auditory and visual attention. There is some evidence that 
this combined activity requires a special study skill. 

18. Listen to sound motion pictures: Both auditory and visual 
powers are involved and the combination may require a 
special study skill. 

C. Oral Activities. 

19. Read aloud. A combination of visual and oral activity. 

20. Engage in group discussion: ihe activity is the reverse of 
listening to class discussion and involves participating in 
the discussion. 

21. Give a special report or “floor talk.” The pupil presents to 
the class a report of a topic which he has previously pre- 
pared. 

22. Participate in a mock trial, a pantomime, or drama. 

23. Engage in a debate; Formal argument and rebuttal between 
two or more srudents in connection with a controversial 
topic. 

24. Give a memorized declamation. 

25. Recite in class. 
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D. Writing Activities. 

26. Write a descriptive theme: (A theme is here defined as 
“continuous written discourse.”) It involves the collec- 
tion, classification and adequate presentation of descriptive 
material in writing. 

27. Write a narrative theme: An item of written discourse nar- 
rating a story or incident, it involves collection, classifica- 
tion, and adequate presentation of narrative materials in 
writing. 

28. Write an expository theme: An item of written disetutrse 
explaining a trend, a point of view, a cause eflTect relation- 
ship, etc. Examples are book reviews and editorials. It 
involves collection, classification and adequate presenta- 
tion of expository materials in writing. 

29. Write poetry. 

30. Write a play or pageant. 

31. Write an advertisement or slogan or newspaper headline. 

32. Make a summary or precis: A summarization or “boiling 
down” of material. 

33. Make an outline or brief: A brief is an argument for or 
against a given point of view. An outline is a rr)pical sum- 
marization of materials properly classified. Distinguishing 
between main headings and subheadings is necessary. 

34. Write a question as for a “question box” or examination. 

35. Write a letter. 

36. Make a list of items: The pupil may be asked to prepare a 
list of the books he has read, the words in a lesson which 
are new to him, important personages, or the functions of 
the Supreme Court. 

E. Drawing Activities. 

37. Draw a true picture: Repre.sent actual objects as accu- 
rately as possible. E.xan)ple; Draw a picture of a mediaeval 
castle. 
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38. Draw a symbolic picture: Express events, objects, or ideas 
in symbolic form. Example: Draw a cartoon showing the 
results of the Civil War. 

39. Draw a diagram or chart. 

40. Draw a graph or statistical table: In addition to tabular 
presentation of data, statistical material may be presented 
in line graphs, bar graphs, circle graphs, and picture graphs. 

41. Draw a map. 

42. Draw a plan, design, or pattern. 


F. (leneral Activities. 

43. Collect materials such as stamps, clippings, and old coins. 

44. Prepare exhibits: A combination of many types of activity 
in a single project, usually on a cooperative basis. Materials 
collected by an individual or group may be displayed on a 
classroom bulletin board or in a special exhibit. 

4.V Construct models, stage properties, etc. 

46. Take part in committee work: Participation in small-group 
work either in the classroom or in extra-curricular activities. 

47. Take part in .school or community projects: Such activities 
as participating in community celebrations, safety cam- 
paigns, school elections, undertaking work in school clubs, 
sharing in community celebrations. 

4H. Participate in the work of .social science clubs, an extra- 
curricular activity. 

49. Preside at a meeting of a class or club. 

50. Take a field trip: An actual visit to a civic institution, 
historical locality, etc. 

51. Make a survey: An inventory is made by the pupil as the 
result of investigating such problems as city planning, city 
housing, or school attendance. 

52. Take a new-type examination. 
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'iJjfiiLJiiumt. (Jtf u. 11. AS ESSENTIAL 

The column on the left contains order of list of skills. The columns which follow 
show the combined judgment of 100 teachers for each grade on the relative import- 
ance of these skills for the respective grade. The numbers in each case refer to 
the item in the first column. 


Item 

Grade 7 

Grade 8 

Grade 9 

Grade 10 

Grade 1 1 

Grade 12 

1 

4 

3 

1 

1 

1 

1 

2 

21 

23 

19 

22 

18 

17 

3 

5 

6 

6 

7 

3 

6 

4 

13 

13 

17 

11 

14 

22 

5 

19 

23 

24 

21 

15 

24 

6 

3 

1 

6 

2 

4 

7 

7 

25 

29 

29 

29 

22 

19 

8 

6 

7 

7 

9 

7 

5 

9 

27 

26 

26 

34 

18 

17 

10 

10 

9 

16 

It) 

10 

8 

11 

17 

17 

14 

17 

20 

27 

12 

16 

13 

11 

15 

12 

19 

13 

19 

19 

18 

20 

25 

G 

14 

1 

2 

2 

4 

4 

2 

15 

49 

41 

49 

47 

38 

47 

16 

49 

45 

37 

34 

41 

35 

17 

33 

36 

43 

27 

38 

38 

18 

44 

49 

45 

44 

46 

38 

19 

25 

26 

26 

26 

28 

38 

20 

1 

3 

4 

3 

2 

2 

21 

14 

11 

10 

8 

9 

8 

22 

36 

34 

37 

39 

so 

47 

23 

44 

41 

39 

34 

33 

30 

24 

49 

45 

49 

50 

SO 

51 

25 

7 

5 

5 

5 

6 

4 

26 

29 

32 

23 

19 

22 

27 

27 

33 

36 

32 

23 

29 

32 

28 

18 

23 

21 

15 

17 

10 

29 

¥> 

51 

49 

SO 

50 

51 

30 

49 

51 

49 

38 

SO 

51 

31 

49 

49 

47 

50 

50 

42 

32 

8 

15 

9 

12 

10 

13 

33 

10 

12 

8 

6 

8 

11 

34 

29 

34 

34 

33 

35 

.'^5 

35 

36 

45 

44 

so 

44 

47 

36 

22 

32 

24 

18 

31 

25 

37 

40 

49 

45 

38 

46) 

43 

38 

40 

32 

41 

44 

37 

47 

39 

27 

18 

21 

25 

20 

21 

40 

23 

23 

31 

33 

27 

IS 

41 

10 

10 

14 

9 

13 

13 

42 

40 

38 

37 

44 

24 

43 

43 

44 

38 

41 

38 

43 

38 

44 

33 

29 

33 

38 

35 

32 

45 

49 

45 

52 

47 

45 

47 

46 

15 

41 

14 

24 

29 

15 

47 

18 

16 

20 

28 

24 

23 

48 

40 

29 

34 

43 

31 

31 

49 

33 

21 

38 

31 

31 

29 

SO 

25 

20 

29 

29 

43 

25 

51 

40 

4! 

39 

43 

43 

41 

52 

10 

8 

11 

14 

15 

12 
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Irem 

Grade 7 

Grade 8 

Grade 9 

Grade 10 

Grade 11 

Grade 12 

1 

10 

7 

5 

1 

1 

2 

2 

38 

34 

29 

35 

25 

19 

3 

6 

7 

2 

11 

5 

7 

4 

2 

2 

9 

9 

10 

13 

5 

42 

40 

40 

33 

27 

39 

6 

4 

2 

7 


4 

3 

7 

44 

42 

43 

40 

38 

34 

H 

10 

9 

S 

16 

4 

9 

9 

40 

41 

44 

42 

37 

33 

10 

10 

15 

20 

18 

10 

6 

11 

25 

22 

12 

14 

23 

21 

12 

17 

15 

14 

11 

18 

14 

1^ 

n 

12 

17 

14 

17 

21 

14 

1 

2 


6 

6 

9 

U 

4S 

49 

5J 

48 

44 

48 

1', 

3 T 

17 

40 

33 

52 

26 

17 

29 

32 

5s 

33 

33 

36 

IS 

'1 

51 

52 

50 

50 

45 

19 

27 


25 

24 

30 

28 

20 

2 

5 

o 

3 

8 

4 

21 

4 

T 

1 


2 

1 

22 

32 

30 

26 

42 

40 

42 

23 

33 

27 

19 

22 

22 

14 

24 

50 

47 

49 

50 

49 

49 

23 

7 


4 

4 

5 

5 

20 

3'' 

34 

24 

24 

25 

29 

27 

37 

30 

34 

26 


40 

2S 

42 

37 

34 

20 

16 

6 

7 ) 

45 

4^ 

50 

48 

50 

51 

30 

3^ 

39 

45 

44 

47 

49 

31 

40 

49 

47 

4o 

51 

46) 

12 

20 

13 

11 

13 

15 

17 

33 

17 

IS 


9 

12 

12 

34 

14 

22 

22 

29 

19 

29 

35 

24 

34 

36 

38 

40 

43 

36 

14 

27 

17 

17 


24 

37 

33 

43 

3'> 

27 

47 

43 

38 

22 

36 

29 

27 

28 

36 

39 

27 

13 

20 

18 

14 

16 

40 

29 

29 

36 

37 

21 

17 

41 

10 

10 

13 

o 

10 

19 

42 

48 

47 

29 

44 

44 

47 

43 

22 

IS 

29 

29 

31 

36 

44 

20 

21 

29 

22 

31 

32 

45 

41 

4<. 

41 

38 

47 

51 

4<') 

17 

18 

17 

22 

20 

23 

47 

17 

25 

32 

41 

39 

31 

48 

16 

44 

48 

47 

33 

36 

49 

27 

20 

22 

31 

38 

26 

50 

31 

25 

33 

35 

35 

26 

51 

52 

51 

46 

52 

44 

41 

52 

10 

11 

14 

6 

12 

12 
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Existing Tests of Skills 

The effort to find tests of skills already prepared which mijjht 
be used in the Investigation was vain. Exercises involving the use 
of skills were abundant. This form of test evidently antedated the 
new’-type question form. Laboratory^ manuals in science, outline 
maps, and map exercises in history and geography contained 
numerous such exercises. Experience had, no doubt, indicated 
a considerable measure of success in their application. I here were 
to be found, also, some efforts to apply the new-type test techni(jue 
to the measurement of skills. Ability to read conventional s\ inbols 
for massed data was being thus tested; and the Rochester Schools 
had devised a test for an even greater range of skills.' After 
extended consideration, it was decided to attenijit the construction 
of tests which should be more economical than the laboratory 
exercises, and should probe deeper or more widely into the skills 
than the existing tests of the ne\v type. 

The Construction of 1'ests of Skill 

Realizing that this Investigation could probably not cover the 
whole range of skills developed or involved in the social science 
studies, it was decided to sample the two main divisions, the 
graphic and the more purely mental. The field of geography 
proved useful for the first and history for the second. Miss Edith 
Parker of the University of Chicago and R. D. Calkins of Mount 
Pleasant Teachers College, Michigan, collaborated on the first. 
Miss Marion Clark, supervisor of social studies, Montclair, New 
Jersey, and E. M. Hunt, Teachers College, Columbia Universits, 
undertook to devise tests of Critical Skills in American Ilistor\ . 
Miss Clark’s experience is also recorded elsewhere in this volume." 
Hunt found tests at the college level so cumbersome a.s to preclude 
the possibility of their extensive use for the purposes of the 
Investigation. 

Trial of the Tests or Skill 

Miss Clark constructed a series of four tests for the elerr.entary 
grades and four for the secondary school grades. 1 hose for the 

*Cf. bibliography, under A. N. Gibbons. 
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elementary grades, upon statistical analysis, were found to satisfy 
the requirements laid down as minimal for objectivity, reliability, 
and grade to grade difFerentiating power, and, with slight revision, 
were approved by the Committee on 7'ests and ordered published. 
I'he critical problems for the .secondary grades demanded more 
extensive revision, which has not yet been completed. Parker and 
Calkins completed the first two of the five or six tests which they 
had planned. I'hese were submitted to trial and are published as 
purely experimental attempts. 1 he tests were not designed to test 
any one program of instruction, but were aimed at the geographical 
knov\ !edge and understanding that the average adult was expected 
to have. 

Reflections REsuLiiNt: from the Trial of Tests of Skills 

1 hese tests have not been sufficiently tried to justify any posi- 
tive conclusions, d heir trial, however, did suggest a number of 
reflections that may be of service to later workers. 

1. It is clear that a distinction must be made between those 
parts of a particular skill which represent what may be purely 
memoriter information and those portions of it which involve the 
practical application of such information to new situations and 
material. 

I he memoriter information, the meaning and relationships of 
conventional sy mbols representing massed data, the technical vo- 
cabulary of geography and social science, the rules of evidence 
employed in social science can all be learned as such. The posses- 
sion of this kntiwledgc can be tested by several forms of new-type 
questions with the same degree of efficiency as definitions or other 
memoriter material. 

1 he practical application of such knowledge to new materials 
presents greater difficulties. 1 he materials to which it is to be ap- 
plied are so voluminous that the cost of the test is greatly in- 
creased, for the material to which the pupil is to apply his skill 
must be included in the test. Another difficulty presented in this 
phase of the test is that relatively little of these skills can be tested 
at the student’s desk. Actual activity on the part of the student 
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is demanded which, of necessity, greatly extends the time involved 
in the test. The combination of larger amounts of material in the 
test and the longer time involved to allow for activities conspires 
to defeat the ends of efficiency essential to the new-type test 
technique. 

2. Another reflection provoked by our experience may suggest 
to later workers a method of overcoming some of these difficulties. 
Supplementary studies including the submission of a simple com- 
pletion type test to seniors in college suggest that the mfmoriter 
material involved in these skills is not retained U’ithovi actual prac- 
tice. In this particular instance, the college seniors were asked to 
complete sentences calling for the name of a particular reference 
device from which certain information could be most efficiently ob- 
tained. These students had made no special preparation for the 
test. On the other hand, the test called for nothing which an in- 
structor might not expect them to know oflTliand at that stage in 
their careers. An individual check upon the findings of this test 
revealed the fact that, though the students had been frequently 
told where they might obtain such information, in high school as 
well as college, they had not been applying thi.s knowledge in prac- 
tice, In this case, the failure to answer correctl\ the questions 
based on memoriter knowledge also proved the lack of practice 
and, hence, the lack of skill. 

Such experience raises the question whether a standardized test 
involving only memoriter knowledge could be made an efficient 
test of a particular skill. Would students, if they were led to ex- 
pect such a test, not proceed to cram this memoriter knowledge 
and thus pass the test with only verbalistic information? "Fhis is 
done by most students with relative ea.se. Perhaps this possibility 
could be safeguarded by wording the question in such a waj'’ as 
to avoid the difficulty. After all, however, the real aim in a test 
of skill is to have the student demonstrate his ability to do. Fur- 
thermore, in doing, the student is expected to give evidence of the 
habit or practice of so doing. Only thus can the teacher be really 
satisfled that the student has gained one of the dynamic outcomes 
of instruction in this field. That, of course, can only be demon- 
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strated by individual performance under competent critical obser- 
vation — a task beyond the reach of new-type test technique. 

3. I'he highly prized ability to evaluate current information, the 
critical ability, is even more complex than the skill of finding infor- 
mation and the skill of employing graphic and conventional devices 
essential to the social sciences. This involves that universally es- 
teemed, but ill-defined, quality of judgment. A person already pos- 
sessed of exact knowledge on a given point is best qualified to judge 
of the accuracy of current information on that point. Such a situa- 
tion, however, is seldom apt to occur. For most, if not actually all, 
people, the reception of new information requires evaluation on the 
basis of probabilities. Value is a relative matter, at least in social 
concerns. It involves comparison with other information bearing 
upon the same point. It is not unlike the practice of the skilful 
shopper who goes about from place to place comparing both quality 
and price. .Applied to social information, judgment depends upon 
the possession of other information and other values to which the 
new may be compared. 

Knowledge of the author’s qualifications for the announcement 
of information may be determined if the author is known and infor- 
mation about him is available. If not, this appraisal requires more 
subtle calculation of probabilities, involving the publisher or publi- 
cation through whom and by which the information is broadcast. 
It can only be conjectured in such a case, but there are degrees of 
conjecture. 

Kvaluation of information may be determined in large part by 
comparison with other available knowledge. Here again, the ques- 
tion of probability enters in, and this, too, is a relative performance 
— the more accurate and detailed the knowledge, the more nearly 
correct the determination of probability. Evaluation of any infor- 
mation, indeed, requires comparison — ^antecedent knowledge of the 
places and persons with whom the information is concerned and a 
delicate determination of the probability of correctness contained 
in the current information. 

In all cases, such evaluation, or judgment, presupposes the knowl- 
edge of a pattern of conduct and relationship constructed out of 
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past information. If the reader or hearer does not possess such 
knowledge, it is necessary for him to turn to the most reliable 
person or critic who does possess it. I'hat pattern, again, is not an 
absolute one, but is constantly in a process of modification and 
reconstruction in light of additional authentic information. 

All these qualifications tend to make the evaluation of new infor- 
mation a highly complex matter which neither a knowledge of 
established rules of evidence nor the possession of extensive ante- 
cedent knowledge can definitely determine. It is never a matter of 
absolutes, but always a relative matter. Under the circumstances, 
the use of the new-type rest, presupposing as it does an absolutely 
correct answer, can have only a limited usefulness in testing this 
ability. Individual perjormancf under competev.i critical scrutiny 
would appear to be the best method of testing this value. Uhat is beyond 
the reach of the new-type test technique. 

4. Skill in expression can only be demonstrated by actual exam- 
ple. Knowledge of rules of grammar does not prove ability to write 
effectively; ability to read a language does not prove ability to 
speak it; ability to interpret pictures does not demonstrate ability 
to draw or paint; knowledge of Robert’s Rules of Order does not 
assure ability to conduct a parliamentary meeting; knowledge of 
economics does not prove ability to conduct a business successfully. 
All skills of expression involve nuiscular as well as mental knowl- 
edge, a knowledge that develops with training and practice. As 
these cannot be acquired without practice, they likewise cannot be 
tested without actual demonstration. 

Perhaps more careful study of the stages in the development of 
such skills will reveal a definite progress similar to that indicated in 
the learning of ideas. If so, it may be possible to confine the test to 
a very limited performance upon some detail of more advanced 
stages in the skill. Successful performance of this detail might be a 
definite proof of all antecedent stages of the skill. The hope of maxi- 
mum efficiency in the te.st of skills lies in this possibility. 

5. In the test of skills, therefore, it would seem that the new-type 
test has only a very limited value. This judgment should not pre- 
clude further efforts to extend those limits. On the other hand, it 
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should warn against reposing too great faith at present in this 
method of determining the student’s attainment of these dynamic 
values of instruction in the social science studies. 

IV. INTKRESTS AND ATFITUDES 

Interests and attitudes are perhaps two distinct topics as viewed 
hy the teacher. ‘ I hej’ are here grouped together because, unlike 
information, under.standing, and skills, both have an obvious emo- 
tional element not so apparent in the first three outcomes of in- 
struction just considered. In the economy of school instruction, 
interest plays an important part as a means as well as an end. 
Attitudes and changes in attitude are of concern there chiefly as 
outcomes of instruction. 

Jntrrrst. Recent pedagogical theory concerned with such methods 
as problems and projects and with the utilization of individual dif- 
ferences has given added importance to the element of interest. 
Research in vocational and scholastic guidance, and in economic 
and social rehabilitation has given further emphasis to this element, 
feachers, whether aware of the activities of pyschology and edu- 
cational theory or not, have devoted much of their energy toward 
arousing the student’s interest. 

An e.\'ploratory investigation by Miss Lena C. \”an Bibber into 
the specific diilicultles encountered by classroom teachers of social 
science subjects affords some evidence as to the importance of in- 
terest in the minds of teachers. “Lack of interest” by the pupils 
was almost universally offered among the first three e.xplanatlons 
for the difftculty of teaching certain specific topics or items. In cit- 
ing this as a common cause of difficulty, the teachers were nor only 
attesting an observed fact, hut were al.so revealing their concern 
about the matter and implying efforts to discover ways of interest- 
ing their students in the topics. 

Analysis of the element of interest for purposes of testing and in- 
vestigation disclosed the great complexity of the problem. The 
psychologists tend at present to look for evidences of “sets” or ten- 

•Pcirv, R. IV, Gfufral Theory of Valuf: Its Meaning and Basic Principles 
Construed in Terms of Interest. New York, 1926. Perry docs not make this 
distinction, but includes both under the Rcncral term interest. 
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dencies, closely related to the biological constitution of the indi- 
vidual. The social scientist is more concerned with the external 
expression of individual interest as that reveals itself in attachment 
to particular social activities or concerns out of the great variety 
with which the individual is constantly surrounded. He may be de- 
scribed as primarily concerned with the possibility of modifying, 
diverting, or even changing the individual’s interest. In this re- 
spect the classroom teacher shares the tendency of the social scien- 
tist and usually views as a deficiency in his teaching skill any 
failure to arouse interest in any topic in the day’s work. 

The discovery of any pronounced individual interest which could 
be utilized by the teacher, as well as the discovery of means by 
w^hich interest could be aroused or diverted for the improvement of 
instruction w^re, therefore, both desirable objects of investigation. 
Pedagogical and psychological literature dealing with these ques- 
tions w'ere found to be abundant. It might almost be said that any 
discussion of methods of instruction dealt directly, or by implica- 
tion, with some phase of the problem, as Dr. Horn’s bibliography 
will show. For purposes of this Investigation, it w as found to be 
impossible to set up a .series of controlled experiments to probe a 
sufficient range of interests. For the sake of economy it was hoped 
that some form of new-type test might be devised which would 
yield a maximum of result with a minimum of effort. It was real- 
ized that this w'as a venturesome undertaking involving great effort 
without any definite assurance of positive results. Dr. Kelley was 
persuaded of the possibility of some success, and his conviction in- 
duced the Commission to sanction an effort to probe for .some fun- 
damental elements of interest as revealed in personality traits. 
With the assistance of M. R. Trabue, a test of such traits by means 
of a free association technique and extended statistical analysis was 
developed.^ In addition, A. M. Jordan carried on an analytical 
study of certain traits by an adaptation of the methods employed 
by May and Hartshorne in their investigation of character traits. 
The report of Jordan's study is included in this volume.* 

•See p. 342 ff. 

®See p. 437 ff'. 
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Any attempt to find some simple, efficient test of the factor of 
interest can only be highly speculative at this time. 7'he sugges- 
tions here offered are based upon a study of the literature and ob- 
servation of the work carried on by the Commission, together with 
a few supplementary studies carried on by graduate students. 

Individual variation in interest in any group of students must be 
accepted as an important conditioning factor in instruction. Just 
how far such individual interest is to be regarded as absolute and 
permanent is still a moot question. The pupil during his growing 
years undergoes important changes in social interest. The general 
trend of personality studies confirms this observation. Recent ef- 
forts to adapt the Strong Vocational Interests Test to sub-adult 
years tend to show that individual interests which are pronounced 
in adult years are markedly less pronounced in earlier years.^ 
Whether this is to be interpreted as indicating that the individual 
has wider potentialities of interest at that earlier age, which circum- 
stances of life rend to narrow and accentuate later on, or whether 
his further development would lead to such narrowing and accen- 
tuation of interest, irrespective of circumstances, can scarcely be 
settled now. While the observatiems offered in this paragraph deal 
with the psychological approach, the search for the fundamental 
organic predisposition of the individual, it must be noted that the 
evidence of such interest is sought in the individual’s reaction to 
external stimuli. 1 hat in Itself raises the serious question as to the 
relative importance of predisposition and external stimuli in deter- 
mining results. Some psychologists would, no doubt, hold that such 
predispositions are definitely contingent upon, and modified by, 
external stimuli, and that, insofar as the theory is correct, such 
rigid separation t)f cause and result is not possible. 

Social scientists and teachers of social science subjects would 
probably all agree that each individual has certain distinct person- 
ality traits. Usually this means a distinct combination of traits, 

‘Frver, O,, Thf Mat .urntifn; -if :»i Rt-Lition to Human Adjustment. 

New York, 19.11, This work l oturuns a amiprcht riiivf survey of efforts to me.isure 
interest and hmlinRS up to the timr it was written. The particular reference 
regarding the adaptation of the Strung Vocational Interests Test to high school 
students is to the w’oik of Dr. Rulon, now at Harvard University. 
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each of which occurs commonly enough in other individuals. The 
effect of such traits is noticeable in the individual’s choice of activ- 
ity. It is only in the most distinctive cases of individuality that 
those personality traits are expected to result in marked differences 
of behavior and activity. The question here involved is whether 
such differences in personality traits are sufficiently sharp in most 
individuals as to justify marked difference of treatment. It also 
raises the question whether most individuals are not capable of a 
fairly wide range in the expression or operation of interest. 

V'iewing the problem of interest from the standpoint of the mate- 
rials of social science, which are concerned with the external stimuli, 
the situation is no less uncertain. There is, first, the difference be- 
tween curiosity and sustained interest. Curiosity is natural about 
objects or situations, recognized as unusual or different from the 
accustomed. Curiosity may be fairly easily aroused about almost 
anything. But curiosity is only the gateway to interest. It may be 
easily satisfied and yield no residuum of interest. 

Sustained interest is in itself a complex matter. To what extent 
is it dependent upon knowledge? Persons interested in games are 
not interested in all games. Is this difference due to the fact that 
they understand some and not others? And is understanding due to 
their interest, or interest due to understanding, or a combination of 
these elements? Teachers have applied duress of some kind to force 
indifferent pupils to study, and then have seen them develop a sus- 
tained interest in the subject. Students who achieve relative suc- 
cess in studies which otherwise have no interest for them have been 
known to continue that interest largely because of their relative 
superiority. Students have been known to avoid subjects in college 
because they developed a dislike for their prototypes in high school 
and frequently, when forced by college regulations to take some 
work in that field, have developed a great interest in it. The reverjc 
has also occurred often enough to be remarked upon. 

The business world presents examples of the operation of interest 
and even more of its cultivation. Sales resistance mu.st be over- 
come. An industry engages in wholesale propaganda to achieve 
that end. Advertising is made as attractive as possible. Articles 
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ranging all the way from cigarettes to automobiles are pictured on 
bill-boards and elsewhere. 'I'hey seldom are pictured alone. Pleas- 
ing colors are used, and the size of the picture is large enough to 
include an alluring background suggesting a bunting scene, or a 
vacation scene, or beautiful young women or men, an elderly man 
in an attitude of comfort, an elderly woman suggesting an idealized 
home — all elements arousing universal pleasant connotations. Or 
the advertisement appeals to a sense of social superiority within the 
price range of everyone. Evidently the merchant is convinced that 
he must make an emotional appeal to arouse the customer’s inter- 
est, overcome his indifference. 

l orn Sawyer’s studied reluctance to allow others to white-wash 
his fence turned a punishing chore into a capital investment of 
considerable proportions. A librarian in a certain liigh school found 
herself deluged by demands for a manual of statistical information, 
never behire the object of any general interest, because a young 
practice teacher had absent-mindedly held a copy of that book 
during the course of the whole recitation. The students were all 
eager to discover what rendered it so precious. These illustrations 
suggest that a sufficient degree of interest may be aroused in al- 
most any enterprise, though devoid of interest in itself, to accom- 
plish the expenditure of both effort and money. 

I hcse illustrations also suggest the importance of emotional fac- 
tors in arousing interest. Whether the appeal is made to the 
student’s pride of achievement, to lus sentimental interests of one 
kind or another, his self-esteem, the prospect of fear or favor, 
punishment or reward, some initial appeal to the emotions seems 
necessary to overcome apathy or indifference. An instructor whose 
good opinion is valued, one who has a pleasing voice, manner, and 
personality, mav gain a degree of interest in his subject of instruc- 
tion from students hitherto indifferent or hostile to that subject. 

It has been said that a teacher can impart any topic provided 
he make it concrete. Such ab.stract topics as the tariff and strict 
and loose construction of the national constitution have been used 
as illustrations. In both instances the exercise was worked out by 
having it involve something of fairly immediate concern to the 
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pupils or their relatives. The materials referred to were such as the 
student had experienced and about which he had definite feeling 
or about which he could readily generate feelings. Here again there 
was an emotional appeal. Furthermore, the term concrete is here 
used in a double sense. It not only refers to the fact that the illus- 
tration is “brought home” to the students, it also refers to the use 
of vivid, tangible objects such as the student might see, touch, and 
feel, or had so experienced. This raises the question whether the 
approach to an idea, an abstraction, or generalization, must be 
through the medium of specific, concrete examples which make 
appeal to several senses and thus afford a substantial ba.se upon 
w’hich the relatively unemotional abstraction may rest. 

If the problem of interest is view^ed as a factor in teaching, the 
great range of activities touched upon by the social science sub- 
jects should afford ample opportunity to enlist some degree of 
interest on the part of every student. History and human geog- 
raphy are as wide as the world in space and time and varieties 
of human activity. Both are rich in concrete material of time and 
place, persons and circumstances. Economics, politics, and sociol- 
ogy deal with special phases of life, but these phases couch the 
lives of all. It would seem possible for every teacher to arouse 
some spark of interest even in the most apathetic student, provided 
the teacher make the necessary effort to bring this material w ithin 
the range of the student’s comprehension. 

Interest as the outcome of instruction in this group of subjects 
is expressed in a great variety of w^ays. The first evidence is the 
student’s performance in the subject. Another expression of that 
interest is added study or reading beyond the bare requirements. 
Still further evidence of interest is afforded by the student’s use of 
his leisure for further reading or study in this field. 1 he choice of 
a career of scholarship in social science would be proof of great 
interest. A further and desirable expression of interest would be 
that adults should make use of social science studies to enrich their 
own activities, w'hatever these may be. And perhaps the highest 
evidence of interest would be that of general, studious, and active 
concern about public affairs. 
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Can such evidence of interest be tested? The performance in 
adult life is obviously beyond the reach of the classroom. A fore- 
cast of such interest can only be approximated. Most of this would 
be determined through scrutiny of the student’s extra-curricular 
activities, voluntary membership in organizations concerned with 
public affairs, the use of his leisure for additional reading, study, or 
activity in this field. And of this only the alert and sympathetic 
teacher can obtain any extended knowledge. Within the classroom, 
the chief source of evidence of interest lies in the .student’s per- 
formances, whether in oral recitation, written essays, or essay type 
questions in examination. 1 he range of interest and the applica- 
tion of that interest in the social science subjects is so great as to 
defy probing by a common test. Of the new-type questions, the 
completion form most readily affords the student the chance to 
indicate his study of anything beyond the common task. Such 
answers would be so varied even in a single class that ouly a com- 
petent teacher could appraise them accurately. Aside from this, 
the attainment of special interest in any field must be inferred 
from relatively higher scores on tests of knowledge in that particu- 
lar subject. 

At this stage in our progress toward the measurement of interest, 
w hether as an element in instruction, or as an outcome of instruc- 
tion, it would seem best to say that we do not as yet have a meas- 
ure. The psychologist’s probing into the fundamental personality 
traits has not advanced far enough to > ield much that the teacher 
can use. 1 he range of interest which the social science subjects 
I can arouse or enrich is so great as to defy any attempt at a common 
or standardized measure. I he importance of interest, both as a 
factor in instruction and as an outcome of instruction, has increased, 
and deserves added, concern on the part of the teacher. 

Jititudfs, Attitudes, like interests, loom large among the ex- 
pected outcomes of instruction in the social science studies and 
have received much attention from psychologists as well as from 
social scientists. It is generally assumed that social attitudes are 
the product of education and experience. Differences of taste and 
interest marking the individual are assumed to find their expres- 
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sion in any society. They may operate to incline the individual 
toward certain social attitudes within that society, but they con- 
stitute only one of a number of factors which determine that choice. 
Education and experience, the latter including the whole complex 
of environmental influences, are other determining factors. 

There is a general belief that the social attitude toward the 
familiar is favorable, toward the unfamiliar unfavorable, whenever 
the familiar and unfamiliar come into conflict or even contact. 
This tendency is presumed to operate with increasing force as the 
individual grows older. Familiarity is, of course, primarily a mat- 
ter of experience. It is also generally believed that education has 
the power to modify attitudes formed through experience and to 
generate or develop attitudes toward social practices, institutions, 
and ideas to which it introduces the individual. 

The term attitude covers a wide range of meaning, only part of 
which falls within the field of social science instruction. Most of 
this teaching may be described as concerned with public affairs. 
Attitudes toward races, religions, nations, forms of government, 
economic systems, social classes and practices, as well as toward 
current social ideas and problems are among the more important 
tendencies usually recogni^ed as falling wdthin this field. 

The term is often loosely used to denote preference, opinion, 
hopes, wishes, judgment, conviction, prejudice, and belief. Perhaps 
these terms express degrees of attitude. At any rate, the term 
applies to the mental or emotional set with which an individual 
approaches a social problem and which determines his line of con- 
duct toward it. It is this dynamic quality which makes the deter- 
mination of attitude so important to government, business, and 
society in general. 

Psychologists have devoted much attention in recent years to the 
study of attitudes. Until relatively recent years their attention 
was devoted chiefly to adults, but in the past ten or fifteen years 
they have been applying instruments of attitude measurement to 
youth and children as well. Efforts have been directed chiefly 
toward studying the effects of various methods and materials on 
changes in attitude. The use of tests of attitude at successive grade 
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levels and in varied curricula promised to yield valuable results for 
the work of the Commission. 

Examination of existing tests of attitude revealed not only the 
vast amount of attention which the matter had received from 
psychologists, hut also disclosed several promising approaches 
which might he used by this Investigation. The attitude-scale 
technique devised by L. L. 'I burstone promised to be most useful 
in the study of attitudes revealed at various grade levels. Other 
forms of attitude test were used in other parts of the Investigation 
concerned with adults. A series of issues was chosen by the Com- 
mission for the development of attitude scales, d'hurstone arranged 
tor a modiheation of the technique of this test to facilitate the 
testing of grf)ups or da.sses of pupils. 1 his procedure is illustrated 
by the following example: 

SrAlhMl.MS tONfl K.NIN'G SOCIAL PROBLEMS 


1-. L. llllRSTONE 
FORM B 

N.irt c Boy or Gir! Date 

Afif at last hirtluLiy Sthool t;ra<lc Scho(>! 


l ollowinn are a titmilxt <jf statenu-nt.s. Some of them will express four views 
and some will not. Plaie a > before ever}' statement with which you 

fiiliv aftrcf. 

1. On the whole, the prohibition laws are satisfactory. 

2. It is better to suffer mdiyinties than to declare war. 

■L I am absolutely against anv re^iulatitm of Sund.ay activity. 

4. 1 do not believe in capital punishment hut it is luit practically advisable to 

abolish it. 

5. Persons wlit' offend .iKainst society deseivt no pity. 
t,frO. riu- lominu.Mist is too radir.il and e.streme in his views. 

7. 1 he C’hinese aie siiperioi to all other rates. 

8. Probibiriori has been a KU.it blessirtR ro our eountrv. 

9. We should make evt i\ reasonable eftotr to asoid war. 

10. bundav Blue laws aie ridiruious in an enliRhtened sueiety. 

11. Capital punishment is Rtwul because it keeps others from committing crime. 

12. Only by extreme punishment can we cute the criminal. 

13. We had better keep our eves on Rti.ssia for a while longer btffore making up out 

minds alwuir communism. 

14. I hate the Chinese. 

1.3, Prohibition should be retained at all costs, 
lb. War i« hardly necessar>‘ in the motlern world. 

17. It is perfectly legitimate to spend Sunday for play. 

18. Capital punishment is the only adequate punishment for murder. 

19. We should abolish all of tiur penal institutions. 
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20. Communists should have the same rights as other people. 

21. There is nothing about the Chinese that I like or admire. 

22. There should be no legal restriction on the sale of liquor. 

23. Peace and war are both essential to progress. 

24. People who want to observe Sunday can do so without restricting other people. 

25. Any person, man or woman, young or old, who commits murder, should pay 

with his own life. 

26. Only humane treatment can cure criminals. 

27. The communists are on the right road. 

28. I don’t like the Chinese. 

29. Prohibition has corrupted our whtJe .American government. 

30. On the whole, wars do the world some good. 

31. Even though Sunday observance is desirable, it should not he compuistpry. 

32. Capital punishment is absolutely never justified. 

33. Harsh imprisonment merely embitters a criminal. 

34. Communistic principles should he adopted everywhere. 

35. I suppose the Chinese are alt right, but I’ve never liked them. 

36. Prohibition should not be forced upon a people whose majority oppipsc it. 

37. Only a coward oppo.ses war. 

38. Whatever else a person does on Sunday, he should go to church. 

39. No thinking individual can believe in capital punishment as a method of 

crime prevention. 

40. Occasional offenders should be reformed. 

41. Our government should he overthrown and a communistic one put in its place, 

42. White people and Chinese can be good friends. 

43. Beer should he allowed under g(»vcrnmeni lirensc. 

44. The highest duty of man is to fight for the power and glory of his nation. 

45. The purpose of Sunday is to glorify God. 

46. Capital punishment is not an effective way to prevent crime. 

47. A criminal should be first punished and then reformed. 

48. Every communist should be put in prison. 

49. The Chinese are a quiet, industrious, peace-loving people. 

50. H the majority of states want prohibition, the minority should agree. 

51. Every last one of us should refuse to take part in any way in any war. 

52. Sunday Blue laws should be strictly enforced everywhere, 

53. It doesn't make any difference to me whether W'e have capital punishment or not. 

54. So-called scientific treatment of criminals docs not stop crime. 

55. Communism would destroy the family and the home. 

56. The Chinese are a great people. 

The application of these tests of attitude was surrounded with 
many difficulties from the outset. The issues involved, the state- 
ments of extreme attitude required by the scale technique, raised 
embarrassing problems which overtaxed the voluntary cooperation 
of even kindly disposed school administrators. It was, therefore, 
impossible to obtain as wide a trial for these tests as for the tests of 
other values. More serious than this, however, was the fact that 
most of the pupils did not understand the ideas involved and ac- 
cordingly failed to register any attitude of significance. The issues 
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themselves were expressed in ideological form, and many of the 
terms used to indicate degrees of attitude proved unfamiliar and 
beyond the grasp of most junior high school pupils. The further 
evidence derived from tests of understanding indicated that it was 
doubtful whether the scales, as constructed, could be used effec- 
tively before the last year of the high school. Other of the diffi- 
cultie.s involved in testing attitudes w-ill be indicated in a considera- 
tion of some oi the problems involved. 

The value of an attitude from the standpoint of the social sci- 
ences lies in its index to social conduct, both passive and active. Is 
an attitude an opinion, a preference, a judgment, a conviction, a 
prejudice, or does it partake of all of these? If the last, atittude is 
not to be regarded as absolute but as a matter of degree. Thur- 
stone’s work indicates that attitudes vary in degree. Closely re- 
lated to this question is that of the fixity of attitudes. Is an attitude 
once formed permanent, or is it subject to change? Does the rela- 
tive permanence of attitude likewise vary, perhaps — to paraphrase 
a concept suggested by Bogardus — according to the emotional dis- 
tance or intensity involved? Are attitudes which the individual 
associates with his sense of self-preservation more permanent than 
those associated only with his self-interest, and are these, in turn, 
more permanent than attitudes associated only with self-esteem 
and thus on to matters of no personal concern? The very fact that 
individuals vary in the degree of their attitudes might he taken to 
suggest a corresponding variation in the relative permanence of 
attitudes. 

Psychologists who have worked with children in the pre-school 
years are convinced that many of the major social attitudes arc 
established before the childern enter school, certainly by the time 
that they enter upon the formal study of social science subjects. 
They have acquired a decided preference in the matter of church, 
political party, social system, some in race, nationality, form of 
government, economic system. Our experience in trying the Thur- 
stone Attitude Scales with pupils in the intermediate and junior 
high school grades raises some interesting questions about the 
nature of such attitudes. How much does such an attitude involve? 
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Let US assume that a small boy has a pronounced attitude favor- 
able to Republicans, opposed to Democrats. The source of the atti- 
tude is clearly the home. The father, perhaps both parents, are 
Republicans. The discussion at the home has frequently indicated 
the feelings of the parents on this point. The boy has come to asso- 
ciate adherence to the Republican party as bound up with his most 
precious loyalties. His attitude, accordingly, is one of strong par- 
tisanship for the Republican party. 

The senior Robert La Follette evidently grew up in such a home 
and early acquired such an attitude. He never left the Republican 
party — even when he became leader of the Progressive group, it was 
the Progressive Republican group. Most of those who were to vote 
for him in his political career had grown up, like himself, with that 
passionate devotion to the Republican party. Yet Senator La P'o!- 
lette had relatively little difficulty in supporting many mea.sure.s 
and principles opposed to those held by most Republicans in his 
day, some of them diametrically opposed. Nor did his followers 
have any great difficulty in supporting him in his stand. In this 
case, the attitude, though deep and lasting, seems to have been to 
the name merely. Similar illustrations could be found for every 
one of these early loyalties and attitudes resulting from them. The 
loyalty is to the name-— not to what the name represents. I hc full 
pattern of the ideology is, of course, beyond the child’s knowledge 
and comprehension. 

The business world affords another illustration. A certain large 
hotel in one of the larger cities has changed its name twice during 
the past four years, though neither the building nor its management 
were appreciably modified. Preceding each of these changes in 
name the hotel had been the scene of a notorious crime widely 
heralded over the country. The notoriety was associated with the 
name of the hotel. Changing the name was thought to remove the 
notoriety, at least, for the casual traveler. Similarly, business 
houses, play houses most frequently, seem to feel that a certain 
stigma becomes attached to them if they retain the same name very 
long. Nearly every large city can exhibit several such houses whose 
names have been changed, frequently with the addition of New. 
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Or a newspaper or periodical identified in the public mind with a 
certain political or economic policy is acquired by a group of the 
opposite political or economic views, and the original name is jeal- 
ously kept. Some business firms have made it a matter of business 
policy to avoid any unpleasant association with the name of their 
firm. They are at great pains to prevent the mention of their firm 
name in connection with any accident involving serious Injury or 
any scandal involving one of their employees. The same principle 
is somi-rimes carried to a ludicrous extreme by zealous promoters 
who seek to establish some connection between their business and 
persons of favorable prominence in the public eye. Favorable or 
unfavorable sentimental associations with a name have a powerful 
effect upon people’s attitudes toward the name. At any rate, busi- 
ness places a high material value on this belief. 

One difficulty encountered in the effort to discover the pupil’s 
attitude was an evident desire to avoid revealing a real attitude 
that he thought would not be approved. This reluctance is by no 
means limited to children, d here seems to be a widespread feeling 
that attitudes arc strictly personal matters to be divulged only 
when, and if, the individual chooses to make them known. Only 
the most rabid partisans are willing to divulge their attitudes freely, 
and for such no test is necessary. A large part of the population 
refuses to divulge its attitudes except under circumstances entirely 
acceptable to themselves. A stranger or an authoritative investiga- 
tion conducting tests is not apt to be included among such accept- 
able circumstances. Here again, however, there is great variation, 
d here appears to be no reluctance about revealing, even to a 
stranger, attitudes on matters of no great concern. But of what 
value is a .show of such attitude which, because it is of no immediate 
concern, may be much changed if, or when, it does become a mat- 
ter of great concern. 

Another obstacle encountered in the effort to test attitudes was 
the operation of what the psychologists have called a defense mech- 
anism, perhaps of compensation. Individuals somehow secretly 
guilty of socially disapproved conduct are at great pains to achieve 
a reputation of the opposite kind. Frequently they are most ex- 
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treme in public profession of an attitude which their own private 
conduct denies. Frederick II of Sicily, the Emperor, is credited 
with the enactment of the most extreme measures against heretics, 
though he was a man of practically no religious convictions him- 
self. The essential value of an attitude as an index to conduct 
would be lost in a verbal or public expression of attitude in such 
cases. With persons of this psychological set a verbal test of atti- 
tude would yield results which must he interpreted as the opposite 
of their apparent meaning. 

Still another difficulty observed in the administration of tests in 
some cases arose from the willingness of the subjects to oblige. It 
was apparent that some of the items in the test presented questions 
upon which the subjects had not spent any previous thought, .^ny 
anstver, therefore, could represent only a flash judgment, the merest 
opinion, which further knowledge might seriously modify. '\’et the 
subjects, pupils in school or adults, as the case might be, answered 
the questions in a way more indicative of a willingness to cooperate 
than of evidence of attitude. Answers of this kind are of relatively 
little value unless critically weighed. 

These difficulties in the use of verbal tests of group attitudes, 
tend to make such testing procedure extremely doubtful in value, 
if not altogether usele.ss, at present. Clerical scoring and statistical 
treatment alone would result in very erroneous impressions. To 
render them at all valuable, the papers would require scoring or 
weighing by some competent person intimately acquainted with the 
group tested. Thus, by far the most certain and valuable attitude 
tests have been those individually administered in psychological 
laboratories, a procedure too costly for purposes of general inves- 
tigation. 

Fortunately, the study of attitudes has been sufficiently pursued 
by many workers and from many points of view to yield some 
observations of value on the question of change of attitude. The 
psychologist with his individual technique, the historian checking 
the completed record of men's utterances and acts, the sociologist 
and political scientist in their studies of social control, the econo- 
mist in his study of marketing, have found an abundance of mate- 
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rial, both on changes in attitude and reasons for those changes. 
Every large social institution, state, church, business, and almost 
every social activity of considerable dimensions has had to deal in 
a practical way w'ith the problem of changing attitudes. Their ex- 
perience, as well as the study of the social scientist, throws impor- 
tant light on the problem of changes in attitude and how it is 
accomplished. 

Perhaps it may be assumed that a person develops some attitude 
toward everything that enters the range of his conscious experience. 

1 he intensity of the attitude, its consequent permanence, is de- 
pendent upon the emotional distance of the experience from the 
individual. 1 he shorter this distance, the more nearly the sense 
of self-interest is aroused, the more intense and lasting the resultant 
attitude. 'I'his emotional distance is conditioned quite as much by 
the character of the individual as by the nature of the experience. 
Psychologists have observed the development of apparently per- 
sistent and fixed attitude.s as the result of an experience that most 
individuals would regard as trivial. The occurrence of the expe- 
rience at a moment when the individual was peculiarly sensitive 
was an important factor in the situation. The extent of the atti- 
tude, whether conlined to a particular place, person, thing, or name, 
or whether extended to involve a whole complex of factors or 
association patterns, is likewise dependent upon individual varia- 
tion, intelligence, and emotional set, as well as upon the concomi- 
tants of the experience. 

As Walter Lippmann has pointed out, the fact that most atti- 
tudes have a verbal cue has been used extensively by agencies 
concerned with social control and business promotion. By little 
more than a change in terminology, legislators, administrators, and 
legal advisers have been able to make palatable policies and 
measures that once w ere regarded as full of unpleasant associations. 
Where the unpleasant associations are inextricably inherent in the 
policy or measure itself, these are bound to reappear, but mean- 
while those concerned with the promotion of the measure will have 
gained their end temporarily, and that may have been their only 
object. Business has made use of this s^ame situation more exten- 
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sively than any of the other social agencies, largely because its 
ends are so often immediate, such as the consummation of a sale. 
It is interesting to observe how frequently publishers or authors 
strive to approximate the title of some book widely and favorably 
known. The result is a flow of style in titles not unlike that in 
fashions of dress. It is clear that a change in name is one of the 
most widely used devices for procuring a temporary change in 
many attitudes. 

Many objects of attitude, however, have names which are rela- 
tively permanent or are associated with cue words which cannot 
be easily changed. Those associated with places, or nationality, 
or race fall in this category. Those associated with established 
institutions, political, economic, or social, present the same diffi- 
culty. Individuals have frequently sought to avoid unpleasant 
attitudes by having their own names changed, They have suc- 
ceeded only where it was their names alone that stimulated the 
unpleasant association and attitude, d owns occasionally and. in 
rare cases, countries have changed their names to avoid unpleasant 
past associations. For the most part, however, large and estab- 
lished organizations are dependent upon other devices than a 
change of name. The process of overcoming an unpleasant attitude 
is a correspondingh' more difficult task. One of the devices much 
used in creating a desired attitude is that of building up a series 
of pleasant associations to counteract the unpleasant. Apparently 
it is easier to counteract pleasant than unpleasant associations, 
favorable than unfavorable attitudes. 

The technique of war propaganda by which the emotions of a 
hitherto peaceful folk are aroused to a ferocious pitch makes use 
of a series of simple emotional appeals constantly reiterated. The 
enemy country is associated in the public mind with violation of 
the most sanctified institutions of family, church, and humanity. 
Stories of inhuman cruelty to prisoners, captives, especially women 
and children, of irreverence toward and profanation of religious 
shrines and dogma, of adherence to political and economic dogmas 
contrary to those esteemed at home have always been fairly 
effective. Not infrequently, extreme partisans on political and 
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economic questions seek to arouse a hostile attitude toward their 
opponents by accusing them of subversive designs upon the same 
sacred institutions. 

The process of changing a deep-seated unfavorable attitude is 
much more difficult. Individuals, once the object of general un- 
favorable reputation, have sometimes succeeded in overcoming this 
by persistent publicity which presents them always in some be- 
nign, humanitarian, and human situation subject to the same 
sentimental concerns which afflict or affect the generality of man- 
kind. The problem in the case of a hostile or unfavorable attitude 
toward some nation is more difficult still. This country had the 
interesting e.xperiencc of being subjected to bombardment with 
propaganda by both sides in the early years of the World War. 
d'he difficulty in this problem is probably due both to the greater 
complexity of the situation and to the fact that there is usually 
a certain amount of counter-propaganda to sustain the unfavorable 
attitude. Oddly enough, a campaign of unalloyed laudatory and 
favorable publicity displa3ing the nation concerned always in a 
righteous pose is unconvincing. A gradual, apparently natural and 
undesigned portrayal of a foreign nation as afflicted with the same 
complex of good and evil, the same efforts to right wrongs and 
achieve justice, subject to the same or similar pleasures and sor- 
rows as one’s own country, human, in short, seems to be more 
effective. 

d he same procedure seems adapted to relieve unsocial race atti- 
tudes, Jbif's Irish Rose served to take the edge off the tension 
reached in racial and religious intolerance, not because it sought 
to represent either race or religion as entirely righteous, but be- 
cause it showed each to have both faults and virtues, to have the 
same fundamental sentimental concerns that affect humanity at 
large. Done with a laugh, it helped to end a situation which had 
become intolerable for most people. If it did not entirely end the 
unfavorable attitude, it helped to restrain it and helped also to 
permit a continued effort to relieve the unsocial aspects of the atti- 
tude. I'he task is a long one and a continuous one in dealing with 
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any large and established organization whose name cannot he 
changed. 

The problem of changing attitudes is complicated by a natural 
tendency to generalize. If each conscious experience begets an at- 
titude, economy of learning and knowledge begets a generalization. 
The individual is tempted to dispose of matters not within the line 
of his interests by classification. Thereafter he can avoid undue 
mental effort by tucking new experiences quickly into some estab- 
lished classification and thus dismiss them from his mind. The un- 
fortunate aspect of this procedure is that, although the individual 
usually follows up an initial experience which is pleasant and thus 
learns more about it, he does not so follow up an unpleasant experi- 
ence which has created an unfavorable attitude. He does genera- 
lize it, however, and make it a classification into which many other 
experiences having only a most casual similarity are quickly thrown 
to share the same unfavorable attitude. This natural tendency 
leads to much social injustice and harm. It can only be corrected 
through the interposition of outside influences which compel the 
individual to acquire added information about the experiences he 
would otherwise dismiss w’ith his too ready classification and un- 
favorable attitude. The school is, of course, the chief agency of 
society to accomplish this result. Added knowledge, acquired as 
knowledge, should, in the natural course of time, result in an in- 
telligent modification of the unfavorable attitude. Such, at least, 
is the hope and endeavor of educators. 

Akin to the common tendency in adults to generalize is the 
tendency of children to form extreme judgments. An act or line of 
conduct is, to a child, either all wrong or all right; a person is 
either bad or good; if a person is once guilty of a wrong, he is cer- 
tain always to do wrong; a person guilty of deceit on one occasion 
is thereafter to be regarded as always untrustworthy. Such judg- 
ments beget attitudes. These are not taken seriously because dur- 
ing childhood family, friends, church, and school are all laboring 
to correct them. Added knowledge is the best corrective. I hus the 
bright pupil, pursuing social science study through his school days, 
should have most of his judgments on matters of public concern 
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adequately modified. On the other hand, the poorer students and 
those who stop their work in social science subjects may continue 
to form such judgments or have their childish judgments crystalize 
into attitudes inimical to a happy society. The tendency to form 
such extreme judgments is corrected only by added learning. Their 
correction will be accomplished only in those fields in which indi- 
viduals continue to work and learn. Since these fields are often 
remote from social science and public affairs, crystallized childish 
judgments may serve to guide them in their discharge of civic 
rights, obligations, and duties. Many important social institutions 
are thereby jeopardized. 

I'he interplay of knowledge and emotion is apparent in all con- 
sideration of the change or modification of attitude. The emotional 
element is paramount in creating or fixing an attitude, which may 
take shape very quickly, even in a moment. The use of knowledge 
to correct or modify that attitude requires long and persistent ef- 
fort. Such knowledge is most effective if it, too, has emotional ele- 
ments. The more concrete and vivid that knowledge, the more 
emotion it will arouse and the more effective it will be in accom- 
plishing the desired modification. Unfavorable attitudes, how'ever, 
die hard; if the emotional intensity is great, doubly hard. Individ- 
uals possessed of unfavorable attitudes have been known, under 
the influence of some emotional experience, to profess a radical 
change in attitude only to revert later to their former attitude al- 
most in its pristine strength. The repentent sinner at the revival 
meeting, the nationality hater after a stirring drama or motion pi 
ture, the persistent party man after a painful exposure of corrup 
tion by his party leaders, all afford excellent examples. A test of 
attitude administered to them at such a time might record a marked 
change of attitude. Such a test, however, would have little value 
in predicting future conduct. Modification of such attitudes can 
be accomplished only by repeated or reinforced treatment extend- 
ing at least through school years and often beyond that time.* 

*Pcrry, R. B., op. cit., p. > 26 , suggests that there are four methods of modi- 
fying attitude or cfTevting "oontrol of interests” which h« summarizes as follows: 

“All of these methods are emploj ed in everyday life— in the waj* of a man with 
a maid or of a maid with a man, in parental exhortation, in religious edification, in 
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The importance of the emotional factor in the formation of atti- 
tude is most fully witnessed in the efforts of church and state to 
establish permanently favorable attitudes capable of resisting all 
modifications. During the past century, the national states have 
displaj^ed the most conscious and systematic effort to accomplish 
this.* Public ceremonials have been increased. 7’he nation is sym- 
bolized not only by flags and statues, but often by some senti- 
mental personification as well. Rituals have been established and 
more are contemplated. Symbols have been multiplied with great- 
ly enhanced enWtional appeal. National celebrations have been 
made occasions of public pleasure and delight. In some countries 
these are accomplished with pageantry in striking colors and varied 
uniforms of public officials. Stories of national heroes, thrilling nar- 
ratives of national history, monuments commemorating heroes and 
national events, the history of the world from the standpoint of the 
particular nation are among the devices calculated to arouse an 
intensity of emotional fixation which no amount of logic and knowl- 
edge can easily overcome. 

The development of interests and attitudes constitutes one of 
the major objectives in social science instruction. None of the ob- 
jectives bear more definitely upon social conduct. There is every 
reason to believe that effective work is being done by the schools 
as well as by outside social agencies, both in the development of 
desired attitudes and in the modification of unsocial attitudes. 
The test of the effectiveness of special devices and methods in 
achieving such results, how^ever, presents great difficulties. The 
desired goal of testing large groups by means of verbal test has 
not yet been attained. Under favorable conditions, the individual 


effrirts. to i-xtt'iid pcrstmal ittfluencr, to obtain public sup[>orr, or to cicatf a piibli<. 
demand. In all of these cases the actepted maxims arc the same. ‘Gisc people what 
they want,’ that is, pit-sent your case so that it will appeal to the ohject’s existing 
interests; ‘Show that you mean it,' that is, manifest in your own person the interest 
which you desire ro awaken in another; ’People value only what they have to work 
for,’ therefore, hold the object at a distance so that an effort is necessary in order 
to obtain it; 'See him just after dinner,’ in other words, cultivate or seize upon a 
receptive mood when the subject is favorably inclined either through his general 
well-being, or through the absence of more attractive alternatives." 

>Sec Merriam, C. E.; 7’Ac Making of Citizrni. Chicagti University Press, 1931. 
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will indicate his attitude on almost any question, but such pro- 
cedures are individual. This fact affords great encouragement to 
the psychologists who are engaged in devising general tests of at- 
titude. 7’he teacher of the social sciences, however, must probably 
be content to await much further work by the psychologists before 
an accurate general test of attitudes is developed. 

V. COMTARISON OF NEW- AND OED-ITPE TESTS 

Nothing has been said thus far about the other forms of test not 
classified as “new-type” or “objective.” The essay type of test 
question has been used, and probably is still chiefly used, by 
teachers in class work and, to a considerable extent, in examina- 
tions. It is used extensively, if not exclusively, in the various gen- 
eral examinations in Luropean countries and forms a large element 
in the College Entrance Board examinations in this country. Be- 
side the essay type question, these examinations also make exten- 
sive use of cxerci.scs and more or less intricate problems, even in 
tests of social science subjects. 

These other types of question were not used in the Investigation 
for a number of reasons. They were lacking in the essentials of 
obvious efficiency. Only experts in subject matter could properly 
grade answers varying as widely in their content and method of 
approach as did such essays. Such experts, if available, w'ould have 
involved greatly increased costs. Eiven with experts, the task of 
standardizing the values of such questions w ould have been practi- 
cally impossible. Ordinarily, no tw’o of the experts w^ould give 
exactly the same, and only seldom similar, value to the answers, 
and the same expert would vary from time to time in his judgment 
of the value of an answer. Though such methods as are used by the 
College Entrance Board to insure uniformity in the grading of 
their papers doubtless serve practical needs, the demands of our 
Investigation could not be thus satisfied. Another deficiency of the 
essay question is the disproportion of time to the range of subject 
matter involved. The limited time of an examination permits only 
a very' few essay questions. The student finds it necessary' to write 
hastily throughout the examination period. He may be fatigued 
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by the writing more than by the thought, and he is so hurried that 
he has usually not time to form his sentences correctly, to say 
nothing of organizing his whole answer in well rounded form. Only 
a small fraction of the subject tested is ordinarily sampled by the 
answers. These three primarily mechanical considerations were 
sufficient to rule out the essay-type question and to minimize the 
use of the exercise- and problem-type examination for purposes of 
this Investigation. 

The fact that the essay, exercise, and problem type of question 
were either eliminated or only slightly used as instruments of inves- 
tigation does not imply that they are equally poor for purposes of 
examination, or for purposes of instruction. Our experience in the 
use of the more mechanically efficient new-type questions, limited 
though it has been, suggests that a comparison of these types of 
questions in terms of the ends of instruction in the social science 
subjects deserves at least a brief consideration. These ends may be 
treated under the categories already employed, namely. (Ij exact 
information, (2) understanding, (3) skills, (4) interests, and (5) 
attitudes. 

Testing Exact Information 

Different groups of subjects vary greatly in the importance of 
precise information as an outcome of instruction. Demanding more 
precise information than literature, but less than language, the 
social sciences, when compared with natural science, logic, and 
mathematics, or technology, appear to require relatively little. 
Formulae in science, mathematics, and technology are learned in 
exact fashion in the classroom and are to be applied throughout 
life in substantially that same fashion. Ideas in social science, 
comparable to such formulae, can never be applied exactly in 
after life. Each application requires some modification, slight in 
some cases and more marked in others. I'he same quality of judg- 
ment which only the scholar in science or the master in technology 
is required to display in somr applications of his formulae, every 
individual must use in all his applications of the ideas of social 
science. TTte reason for this lies in the fact that every social situa- 
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tion is highly complex; that it contains unique elements or com- 
binations of elements; that it is never the same as any previous 
situation — persons, place, or attendant circumstance usually, and 
time always, being different. The individual applying his social 
idea is always required to estimate how far the diverse and unique 
elements will necessitate modification of the idea. The scientist 
often, and the technologist nearly always, is able to control his 
material, so that his formulae may apply as learned, but the social 
scientist is practically never able to control the situations to which 
his ideas apply. 

Whatever exact information the teacher expects the student to 
learn, whether this consists of specific persons, places, events, cir- 
cumstances, definitions, conventional symbols, typical relationship, 
established procedures in gathering information or rules of evidence, 
is properly the subject of test at intervals during the year as well 
as at the end of the work. This information has the character of 
precision and definiteness. The answers are either right or wrong 
according to a standard or model furnished in the instruction. 
Both the new-type and essay form of question have been abund- 
antly used in testing the student’s possession of this knowledge. 
For this type of material, the essay question is cumbersome, >vaste- 
ful of the student’s time and the teacher’s patience, and usually 
unfair to the student. If the teacher is forced to hunt through 
several pages of hastily written, almost illegible, scrawl for a few 
items of exact fact, it is doubtful whether the teacher wdll find them 
all, and, if the number of papers to be graded is large, it is certain 
that the teacher will not find them all. And what a waste of the 
teacher's energy in marking and of the pupil’s energy in writing 
that small bundle of facts becomes! If proper and intelligent care is 
exercised in the construction of new-type questions, the teacher 
can test the student’s knowledge of many times the number of 
items in the same period of time required by one essay answer. 
These answers w'ill sample a much greater segment of the subject 
and therefore more correctly test this part of the student’s attain- 
ment. And, in addition, this form of test will have involved much 
less fatigue for both teacher and pupil and greater accuracy in the 

[ 97 ] 



TESTS AND MEASUREMENTS 


scoring. The comparison of the two types of test for the measure- 
ment of achievement of precise information is so decidedly in favor 
of the new-type test as to make the continued use of the essay form 
for this value seem a decidedly outmoded procedure. 

Before accepting this result, however, a few caveats must he 
taken into account. The new-type test must be carefully construc- 
ted (1) to minimize the element of pure chance in the answer, and 
(2) to avoid ambiguities, thus insuring the student’s unequivocal 
reaction to the true purpose of the question. The help of someone 
experienced in test technique is desirable. To insure the student’s 
honest performance, the new-type test must be administered with 
even greater care than the essay type. Seating arrangements, 
varied forms for the same test, and efheient proctoring must be 
carefully considered. If circumstances require a modification of the 
test to be given at intervals, to insure an accurate comparison of 
the student’s achievement, the several forms of the test must be 
weighted to insure reliability. This entails statistical treatment. 
Experience has shown the importance of these caveats. If they are 
not respected, if the teacher attempts to dash off such a test in a 
short time and administers it in a casual fashion, few of the ad- 
vantages will be gained, and the incentive to dishonesty will be so 
increased as to outweigh even these few advantages. Such cautions 
cannot, therefore, be too greatly emphasized. 

Testing Understanding 

The understanding of ideas is generally considered to be a much 
more Important outcome of social science instruction than any 
body of specific information. Ability to apply the idea correctly to 
a new or unaccustomed situation would usually imply understand- 
ing. Ability to select from a group of related ideas the one which 
most accurately meets a given situation w’ould likewise indicate 
some understanding. Ability to trace the probable ramifications 
or relations of an observed social phenomenon would also afford 
evidence of understanding of the idea involved. Mere ability to 
repeat a definition of the idea vcould be unconvincing — especially 
if repeated in the exact words of text or instructor. It might be 
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then merely a verbalism having no more meaning than any other 
combination of words. The essay-type test has long been used, 
with varying degrees of reliability, to test understanding. The 
student’s extended discussion usually offers sufficient latitude to 
reveal not only the understanding of the idea but also the degree 
of that understanding. If the new-type test could also be applied 
to this end, it would afford the teacher an opporutnity to test a 
much greater range of ideas in a shorter time and thus afford a 
fairer and more complete indication of the student’s attainment of 
understanding. The discussion of the Wesley, Kelty, and Pressey 
tests has indicated that this type of value can also be reached to a 
very appreciable extent by means of the new-type test. Discrim- 
ination between related ideas, ability to apply known facts to a 
new situation, and even some test of the student’s ability to trace 
relationship, have all proved po.ssible. The construction of such 
tests, however, is much more difficult, the possibilities of standard- 
izing such tests less certain. It is not clear that they can be made 
to supersede the essay type of question for this value, but it is 
clear that they can he made a very helpful supplement. 

Testing Skills 

The scholar’s mastery of the skills of his calling is demonstrated 
in the successful execution of a thesis. This is comparable in the 
field of learning to the masterpiece of the craftsman in the manual 
arts. Most pupils, however, stop short of the higher degree in any 
of the social sciences. For the earlier stages such devices as a course 
or term paper, an individual topic, afford the student opportunity 
for the use of skill. All such devices might be described as exercises 
demanding the individual use of skills involving both knowledge 
and practice. They require extended time. Whether the new-type 
test can be developed to afford a more rapid, yet accurate, test of 
the student’s possession of the skills remains to be determined. 
The two tests developed for this Investigation, the one on geog- 
raphy and the other on historical criticism, vs^ere essentially 
exercises which made use of new-type questions. Insofar as skill 
involves a certain body of precise knowledge, new-type tests can 
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be developed to test such knowledge. Insofar as skill is a matter of 
practice and use, it seems doubtful whether the new-type question 
can be of much help. The hint contained in one of the exercises 
mentioned under skill, that the knowledge essential to a skill is 
not retained except through practice, might suggest that the new- 
type test could be used fairly far in testing skill. A test of the 
knowledge applied sufficiently long after that knowledge was im- 
parted might, if the knowledge were still present, give adequate 
evidence of the use of the skill. Much additional work, however, 
will be needed before the new-type test technique can be used 
except as a supplement to exercises. 

There is one skill which may, indeed has been, seriously impaired 
by the excessive use of the new-type test. This is skill in expression, 
the ability to set forth some topic in social .science clearly, con- 
vincingly, and agreeably. Teachers in college have begun to re- 
mark that students who come to them from school systems in 
w'hich the new-type test has been used almost exclusively for a 
number of years are unable to express themselves cogently either 
orally or in writing. In school systems in which the new-type test 
has been used extremely, it has been possible for students to avoid 
writing a single complete sentence except in courses in Engli.sh 
composition. Inasmuch as coherent and cogent composition is still 
one of the most widely used, as it is one of the most valuable, 
skills in social science, it would seem essential to continue to use 
the essay-type examination, if for no other reason than to afford 
practice in this skill. 

Testing Interests 

The discussion of interest as an outcome of instruction indicated 
that interest was strongly conditioned by the individual’s emo- 
tional set. A test seeking to discover the evidence of interest in a 
given class would, therefore, have to provide for a wide range of 
interest. Such a condition would defeat the ends of efficiency 
which particularly recommend the new-type test. The essay-type 
question, which, by its very elasticity, permits the student freedom 
to reveal his thinking and reading, is probably the most economical 
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method of discovering evidence of a sustained interest developing 
out of the work in the subject. This type of question especially 
invites the creative and constructive interest of the good student. 

A proper appraisal and appreciation of this evidence requires the 
competent judgment of an expert — teacher, scholar, or both. It 
also involves the conscious employment of the essay-type examina- 
tion as a test of interest. 

Testing Attitudes 

The pitfalls of a direct effort to test the attitudes of students on 
questions which they regard as important and about issues of a 
controversial nature are so many as to defy any form of test. The 
new-type test is most direct and perhaps, therefore, least efficient 
in gaining results. The essay type affords the student an oppor- 
tunity to reveal his attitude on many questions. The methods used 
by the historian in checking act against verbal statement may still 
be the surest method of determining real attitudes. Next to that, 
the essay type, as interpreted by the teacher, would promise to 
yield most satisfactory evidence. 

The Examination 

The function of an examination given by a teacher at the end of 
a course to students with whom he has been working over an ex- 
tended period of time may be purely ceremonial and is never more 
than a supplementary index of the student’s attainment in the 
work. On the other hand, the function of an examination by 
strangers is to afford to the latter a faithful index to the student’s 
total ability and attainment in the field. Though superficially sim- 
ilar, these two examinations are widely different in fact. 

I'he latter, the examination as a true and faithful index to the 
whole achievement and ability of the examinee, is becoming in- 
creasingly important. The growth of civil service,* the develop- 
ment of personnel departments in industry, the use of qualifying 
examinations for admission to higher stages of learning, have all not 
only increased the number of such examinations but have given 

•Anderson, W., and l.anibie, M. B., Univfrrity Training for tkr Xttiiona . 
Srrvue. University of Minm-sota, 1931. 
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added importance to them. These developments have made it cor- 
respondingly necessary that the examinations used for this purpose 
be proper examinations. 

The high hopes for a speedy, comprehensive, and accurate exam- 
ination which were aroused by the application of the new-type test 
to measurement of learning have not yet been realized. This tech- 
nique has proved efficient in testing precise information. It, there- 
fore, has great value in examinations in technology, considerable 
value in the lower reaches of science and language, but as yet 
relatively little in social science subjects, primarily because the 
body of precise information in these subjects is of relatively less 
importance. Such experience as we have had in attempting to rest 
the other values indicates that there is further possibility of im- 
proving the examination and of using the new-type test in that 
improvement, but there is much preliminary svork still to be done. 
It is apparent that the improvement of the examination must re- 
ceive the attention of social scientists as well as psychologists.^ 

Meanwhile, until the necessary' further work has been done, it 
seems desirable to caution those who are not expert against the 
extended use of the new-type questions for values other than that 
of specific information. Such tests are, of course, valuable in deter- 
mining growth in social science vocabularies. The essay and exer- 
cise types of questions scored by' competent teachers will he more 
serviceable until those further developments have occurred. 

Conclusions 

The study of testing in the social science studies will doubtless 
continue. 1 he work of the Commi.ssion must necessarily end. Any 
conclusions offered at this time must be regarded as an inventory 
of progress at the point where the Commission ended its labors. 
No finality is or can be claimed for them. Further work will cer- 
tainly modify them and may prove some of them wrong. 1 hey are 
presented here as our best present judgment. 

*It is interesting to note the advance which has been made where snch co- 
operation has been systematically applied, e. g., the test in United States history 
in the fifth annual Iowa academic contest, 1933. 
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1. Are Tests Necessary? In the past generation it has been seri- 
ously questioned whether the test or examination is a necessary or 
desirable adjunct to instruction. The main arguments of the dis- 
cussion for and against the examination are briefly summarized in a 
recent work' and need not be reconsidered here. The following 
observations may be added. It is not likely that pupils will cease to 
desire to submit the evidence of their learning at intervals, both to 
satisfy themselves on the certainty of their progress and to gain the 
friendly, helpful criticism of those who are masters of the learning. 
It seems no less likely that conscientious teachers will continue to 
reassure themselves b}' test of the satisfactory completion of one 
stage of the work before entering upon the next stage. Some form 
of periodic testing will probably continue to be regarded as an es- 
sential element in instruction wdthin the classroom. Outside the 
classroom, the demand for testing has greatly increased in recent 
years. I he growing centralization of school systems has accentu- 
ated the importance of the test as an inventory of educational 
efficiency, not only in the minds of the school administrators, but 
also of the public. I'he tendency of colleges to limit their enroll- 
ments, whether as a matter of total enrollment or for admission to 
more advanced or specialized work, has given added emphasis to 
the test as a selective device. 1 his development has led some col- 
lege administrators to seek the perfection of tests which can be used 
for guidance purposes to help the student determine his own course 
with the least amount of waste.- Perhaps even more important has 
been the increase in civil service and the development of more 
searching selection of employees for industry. Testing has thus 
been extended to the world of affairs, giving added emphasis to the 
testing done in the schools. These observations add heavily to the 
conclusion of the writer cited that testing has become more, rather 
than less, important. It is riot a question whether there shall be 

'loni.'s, F.. S., ('r.m^r.-hrnnri’ in .{rturrienn New York, 

Mactnillan, C't. kellev. 1. L., 7’Af Scirnttfic Method in /Education. Revised 

Keiition. 1052. 

’Paper read hr’ Dean j. H. Johnston of the University of Minnesota at a 
meeting of the ,\inerican Council on Education and the Educational Records 
hureau, New York City, November 3, 1932. 
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tests, but rather how these tests may be more effectively adapted to 
discovering whether purposes and ends — objectives — have been 
realized. 

2. Are Existing Tests Adequate? The growth in the use of testing 
has been accompanied by a scrutiny of tests used. The customary 
examinations derived from classroom practice were assailed as un- 
reliable, often invalid, and highly inefficient.* They were criticized 
as inadequate to the objectives of instruction, and their grading 
was denounced as highly inaccurate and subjective, scores fre- 
quently being determined by irrelevant personal preferences of 
those grading the examinations. This criticism was so well founded 
and mounted to such proportions that some institutions of learning 
began to make even their general examinations a matter of sus- 
tained study.^ The new technique developed by psychology, with 
its obvious advantages of efficiency and accurate scoring, com- 
mended itself rapidly to the industrial world and appealed strongly 
also to educational administrators. It seemed so superior to the 
traditional examination procedure that it threatened to replace the 
older method entirely, and psychologists were invited to develop 
tests for nearly all school subjects. Standardized tests employing 
this technique and fortified by statistical studies of reliability and 
validity came into general use. 1 hese, t(X), have now' been used 
long enough to disclose weaknesses almost as serious as those found 
in the older procedure. 7 he conclusion that existing tests are not 
adequate is one of the most definite conclusions derived from our 
experience.* 

3. The Relation of Tests to Objectives, Analysis of objectives, 
whether of all of the social science studies in toto or of any one of 

*The literature criticising the traditional examination is now quite voluminous. 
See appended biblic^raphy. 

“Harvard and Swarthmore are pioneers in this field. The University of 
Chicago is engaged on a very extensive program at the present time. 

“Perhaps the best brief summary of the deficiency of existing tests from a 
technical standpoint is the paper prepared for the American Council on Education 
by E. F. Lindquist of the University of Iowa. This paper was based upon extended 
study and is supported by a number of graduate theses, among which is an unpuh- 
ishcMi doctoral dissertation hy E. C. Denny, An Investigation of the Defects and 
Weaknesses in Certain Objective Tests in American History. University of Iowa, 
1932. This has special interest for this field. 
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them, disclosed certain definite types of objectives, each type in- 
volving important divisions as the objective was considered in con- 
nection with specific material. The types of objectives may be 
indicated as follows’ 

A. A body of information 

B. A body of ideas accompanied with special vocabulary' 

C. Ability to apply ideas to new situations 

D. A body of skills 

E. The development and acquisition of interests 

F". The acquisition of certain social attitudes 

G. Ability to express social learning clearly and effectively 
While each of these types has many subdivisions which vary ac- 
cording to the particular subjects taught, the types occur in all of 
the subjects. .Any examination purporting to test learning in this 
field must take at least these types of objectives into account, how- 
ever much it may vary in detail. Few', if any, of the existing exam- 
inations do so. The new-type examinations with their strict limita- 
tion to the material set forth on the test papers may be more 
deficient in this respect than the old procedure which, by its very 
elasticity, permitted the examinee to reveal much more learning 
than was called for by the questions. Both, however, are subject to 
serious criticism for failure to consider the range of objectives. ‘ 

4. The Interrelation of the Types of Objectives in Social Science 
Subjects. The objectives in .social science subjects have a definite 
relation to each other. All are related to information. Ideas are 
generalizations dealing with specific facts, whether as descriptive 
of known and studied facts or as applied to new facts. Skills are 
skills in dealing with information. Attitudes, and here we are re- 
ferring always to cultivated attitudes, are attitudes toward certain 
facts or bodies of information. Social information constitutes the 
focal nexus of all objectives. While this is true, the order of rela- 
tionship is the opposite of the obvious. The possession of informa- 
tion does not assure the possession of the other objectives. The 
possession of the cultivated attitudes would more nearly' imply the 

•Ofburn, W. ]., Are We Makine Good in ike Study of History. Public School 
Publishing Company, Bloomington, tlUnois, 1926. 
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possession of the information. So, too, would the possession of the 
requisite skills. The aim of efficiency in testing, namely the goal 
of testing as much as possible in as little time as possible, is not 
served by a test of information alone. The number of items of in- 
formation wdiich can be tested in any given time is relatively 
small. These items can easily be committed to memory without 
acquiring any of the other objectives. The process of acquiring the 
other objectives is a laborious one involving much information and 
reflection about the relation of that information. The process of 
gaining the information is essential to the acquisition of the skills, 
but skills themselves are the result of much practice in this pro- 
cess. Testing for the objectives of instruction in the social science 
subjects must take this relationship into account. 

5. Suitability of Different Types of Question to Test Types of Ob- 
jectives. The different types of objectives vary greatly in definite- 
ness. Information is most definite, attitude least so. 

A. The possession of information* can be best tested by means 
of the new-type test technique. I his type of question can search 
out a greater range of information in shorter time and be marked 
with greater accuracy at less cost than can the essay type. Any of 
the forms of the new-type question may be used for this purpose, 
multiple-choice, matching, completion, rearrangement, even the 
true-false form. Those questions having the greatest definiteness 
and the least opportunity for chance are to be preferred. 1 he com- 
pletion form is least definite; the true-false form offers the greatest 
opportunity for pure chance performance and is, therefore, least 
desirable. 

'Those who have followed the discussion closely up to this point may have 
been somewhat troubled by apparent changes in the distinction between information 
and other forms of knowledge which have appeared at various stages in the progress 
of the work. In a sense, these changes have faithfully reflected the growth in appre- 
ciation of the problem involved. I hc distinctions are not universal, but arbitrary. 
As arbitrary distinctions they have a practical value in the teaching of the social 
science subjeers, which is the only justification for their employment. The 
differences involved upon which the practical distinctions have been made are 
matters of degree rather than of kind. 

As here used in the concluding portion of this discussion, information refers 
to the materials of the social world, persons, places, and objects of various kinds 
and the terms by which they arc identified or described. The external circum- 
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B. The possession of ideas, so far as this involves merely vo- 
cabulary and fixed definitions, can also be best tested by means of 
the new-type test forms. The same considerations apply to the 
selection of forms as in the case of the test of information. 

C. The ability to apply the ideas to new situations can be only 
partially met by the new-type forms. The ideas vary in complex- 
ity and the situations calling for their application involve varying 
amounts of description, some beyond the scope of the short- 
answer form. Of the new-type forms which may be used, those 
calling for a high degree of selection are to be preferred. The four 
or five-option best-choice form of questions, in which each of the 
options bears some relation to the question, proved most efficient 
fi»r our purposes. 

sr.'UK'o'i or t'vcnrs ma»' also he iiicluclcd. Information, so considered, is descriptive, 
referrinn to separate and static materials. 

IJfi3 hero refers to the relationship of the materials of the social world, 
actual and potential, their purposes and uses, the problems and processes of their 
relationship. .‘Vny instininons, principles, laws, and ideals of society are, of course, 
inchulcd in this catej'ory, 

A distinction of practical value to the teacher of social science subjects may be 
applied to both ii.f'irmati in and idea as here dehned. .All of the information, and 
so much of the idea as consists merely of verbalisms designating or describing 
ideas, institutions, verbal formulae, laws, is only potentially valuable to instruc- 
iion and learning m this held. Such knowledge conslilutes the means but not 
the end or purpose of instruction. If the learning is not carried beyond that 
stage It is, .so far as the teacher of the social science subjects is concerned, “inert.” 

'I'his distinction is also arbitrary and perhaps of practical concern chiefly to 
the teacher of these subjects. Such material is an essential preliminary to the 
“fiinctioiia!” knowledge with wliich the teacher is most concerned. Perhaps in an 
absolute sense such knowledge is not “inert,” the difference being again a matter 
of degree. Most teachers of these subjects, however, would regard their efforts 
as unavailing, if their pupils possessed only .a list of names and verbal formulae 
without realizing the interrelationship of the materials and ideas which these 
n.imcs and verbal formulae designate. I’he distinction, therefore, must be regarded 
as practical rather th.m philo.sophic. 

It i.s probably true that no item of information which the pupil acquires is 
completely “inert.” It i.s equally probable that no one ever attains complete func- 
tional knowledge of social information. Tbe process of endowing what the teacher 
calls “inert” knowledge with its functional attributes is a gradual and continuous 
one. It proceeds more rapidly in the caseofiiifunnation in which the pupil is vitally 
interested and less rapidly where the interest is little or non-existent. The per- 
functory acquisition of information in the latter case may consist largely of “inert” 
material. In any c.i.se, the point in the process of developing information from the 
almost inert to a distinctly functional knowledge is not easy to establish. It may 
be less a point than a span, before which the inftirmarion has seemed inert, and 
after whicn it is recognized as functional. It is this functional addition to informa- 
tion which the teacher recognizes as understanding. 
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D. The possession of the essential skills calls for performance 
in addition to knowledge. Where the performance can be accom- 
plished by means of an exercise of small compass, the new-type 
forms of question may be used to supplement the exercise. In this 
case even the true-false form may be used efficiently, where the 
intermediate operation of the exercise is assured. 7'he more ad- 
vanced stages of skill, however, require a scope of performance 
beyond the technique of the new-type forms and demand the ap- 
plication of expert judgment to insure the best appraisal of the 
performance. 

E. The evidence of developed interests, which are matters of 
both degree and kind and are therefore less tangible than any of 
the preceding objectives, is less easily discovered by means of the 
new-type forms of question. Of the new-type forms only the com- 
pletion form affords a sufficient range of answer to sample both 
kind and degree of interest, and this requires expert appraisal. The 
essay form of question affords still greater elasticity and it, too, 
must be appraised by^ expert judgment. 

F. The pos.session of the cultivated attitudes is least tangible 
and most elusive, and the test of their true presence is afforded 
perhaps more often by indirect or incidental evidence than by 
direct. The ordinary forms of the new-type test are too direct to 
be serviceable. Whether psychological research will be able to de- 
vise some efficient instrument to test this objective remains still 
to be determined. The essay form with appraisal by expert judg- 
ment seems at the present the most certain form of test for this 
objective. 

G. The ability to express social learning and understanding, 
clearly and effectively, is a matter of performance orally or graphi- 
cally. The written essay or exercise appraised by expert judgment 
offers the best test of this objective. The use of the short-form 
type of test for this value is definitely objectionable. 

6. Modification of Tests to School Grade Learning. Those con- 
cerned with the testing of learning in the social science field are 
fully developed adults accustomed to the use of that learning on 
an adult plane. Though psychology has relatively little definite 
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guidance to offer, there is reason to believe that social learning 
follows some order of development and growth. I'he obvious bio- 
logical development of the child suggests important changes in 
attitudes and interests. In this period he is transformed from a 
condition of complete social dependence to one of apparently com- 
plete social independence. From one who seeks guidance, he be- 
comes a person who tends to rely entirely upon himself. These 
facts strongly condition social learning. Our experience in apply- 
ing the same tests to successive grades in school suggests certain 
phases of progress in social learning which later research, it is 
hoped, will specify more closely. 

A. The accumulation of information is a progressive process. 
Allowance should be made for this fact. On the other hand, where 
the information is organized, as most materials of instruction are, 
the main facts are most thoroughly learned and progress is marked 
by the addition of subordinate information. If the time for test- 
ing the pupil’s store of information is limited, the test should con- 
sist primarily of subordinate items in various degrees of detail. If 
the pupils know the latter, the knowledge of main items may be 
assumed, while the reverse is not true. A test made up chiefly of 
main items is, therefore, not a test of the accumulation of infor- 
mation. 

B. Ideas are learned progressively. This is just as true of a 
single idea as of a store of ideas. (3nly the very simple ideas are 
mastered with precision in the earlier grades. Imperfection is to 
be expected and the questions used in the test, if of the new-type, 
should provide for degrees of precision. 

C. Ability to trace relationship is the product of teaching as well 
as of study. Aw'areness of, and ability to trace, multiple relation- 
ship is a slow development and not to be expected except as the 
result of definite training. In the case of highly organized activi- 
ties, such as those of a great government or industry, pupils may 
be led relatively early to recognize the fact that anything aflPect- 
ing the center of the organization will sooner or later affect all 
parts of the organization. In the case of unorganized activities, 
such as that of a new invention, the recognition of the full sweep 
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of its relationship requires both knowledge, imagination, and train- 
ing. The public official who was able to profess adherence to the 
slogans that the interdependence of the world should be recog- 
nized, that national industries should be protected, and that Smith- 
ville money should remain in Smithville affords an interesting 
illustration of a lack of such training in recognizing multiple rela- 
tionship. Limited results are to be expected from pupils still in 
school, but these can be tested by essay questions. 

Organized institutions, governmental, economic, or social, all 
represent multiple relationship, the larger and more highly organ- 
ized, the more numerous and extended their relationships. The aim 
in instruction is to have the pupils appreciate this multiple rela- 
tionship. I'he step by which the pupil advances to that learning 
is through the concrete and visible appurtenances of the institu- 
tion. 7he material equipment, buildings, the personnel, relation 
of personnel to material objects, relation of personnel to other per- 
sons, first in single relationships, and after that in widening rela- 
tionships, appear to enter the field of the pupil’s appreciative 
knowledge in about this order. Knowledge of the material char- 
acteristics of an institution can not be used to imply knowledge of 
the institution. Knowledge of .some of the relationships of the in- 
stitution implies much more, but is still inadequate. Knowledge 
of the institution is not fully satisfactory' until the pupil can trace 
the multiple relationships of the institution in at least its more 
important economic, political, and social phases. Such knowledge 
can only be approximated during the public school years. The 
multiple relationships of unorganized and sporadic elements in so- 
ciety are more difficult to trace, since imagination and thought as 
well as learning are involved. The thought concepts behind insti- 
tutions, the pattern of ideas involved in the dogma or ideology, 
represented usually by words ending in -ism, are more difficult 
still, because they lack the concrete material and the activities of 
the institutions themselves. The notion of process, which regards 
institutions and ideologies as passing devices through which so- 
ciety is seeking to satisfy its fundamental needs, is most difficult. 
During school years, therefore, the tests should be designed to 
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measure the degree of such knowledge of relationship, apprecia- 
tion of each additional relationship marking desirable progress. 

D. The acquisition of skills is not separate but incidental to the 
growth of knowledge. The simple prototypes recognized and cul- 
tivated in the earlier grades become increasingly complex as the 
pupil builds up the store of his knowledge, both of facts and their 
relationship. The additional refinements, though recognized under 
separate names, owe their value to the soundness of the foundation 
of antecedent work in skills. Tests of skills, therefore, require 
provision for degree of mastery. This is usually afforded most 
easily by offering degrees of difficulty in the material upon which 
the pupil’s skill is tested. 

K. Interests are conditioned by a number of factors including 
individual inclination, biological development, personal experience, 
and learning. Assuming that the chief duty of instruction in social 
science subjects is to cultivate interest in public affairs — those 
social concerns which affect the group as well as the individual — 
the conditioning factors exercise an important effect. Lacking 
personal responsibility, the pupil’s interest is correspondingly 
difficult to develop. His economic dependence, his physical and 
social immaturity, make it difficult to advance far in the cultiva- 
tion of a well-rounded interest in all phases of public affairs. There 
is enough similarity between his personal group-life experiences 
and the political affairs of more adult society to develop an interest 
in this phase of public affairs somewhat earlier. The development 
of interest in the other phases must necessarily lag. even though 
teachers seek to stimulate that interest by vivid, concrete experien- 
tial material. Evidence of interest detected in incidental revela- 
tions in the pupil’s work should be fostered. 

F. Attitude.s and judgments might appear to undergo an oppo- 
site de\'elopmenr. Positive and decisive at first, they are gradually 
mt»dified under the influence of Increased knowledge. As in the 
case of interest, one of the first concerns of the social sciences is 
to develop attitudes and judgments on public affairs. The attain- 
ment of attitudes and judgments based upon well-rounded knowl- 
edge of society is the goal of that instruction. Teachers may assume 
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that their primary function is to help develop that knowledge, and 
with more or less assurance that, if that task is well done, the atti- 
tudes and judgments will take care of themselves. Under those 
conditions, the possession of the socially desirable attitudes is not 
to be expected until toward the close of formal instruction. The 
evidence of extreme judgments and attitudes in the earlier years 
is to be expected, their Improvement a matter of gradual develop- 
ment until the pupil has acquired a wider or deeper knowledge of 
society as a whole. 

7. The Possibility of Developing Standardized Tests. The obvious 
interests of efficiency would suggest the desirability of standardized 
tests equally useful in any part of the country year after year. 
Statisticians have devised a method for such standardization. 
However, even the construction of a test of factual information, 
to be valid as well as reliable, is an expensive undertaking much 
beyond the resources of the individual teacher or subject-matter 
department. That fact alone would make it desirable, once a test 
had been made, to continue its use widely and for an indefinite 
time. It remains to consider the factors involved. 

A. Specific information lends itself most easily to the new-type 
form of test, and hence to necessary statistical treatment. The 
amount of specific information which may be used in the social 
science studies, however, is so great that only a small portion of 
it can be used in any single course. Localities will inevitably draw 
upon their own resources for some of this information. The pupil 
learning about society in London will do so through a different set 
of facts from those learned by the pupil in New York. The pupil 
in Boston will learn many facts w'hich the pupil in California will 
not learn, and vice versa. If it were possible to construct a test 
limited to those specific facts which all four pupils learned, it 
would consist of few items and those only the general items. Thus 
it would cease to be a test of prepress or a test of the extent of 
information. If a test of specific information which did measure 
extent of information were devised, it would have a maximum 
efficiency in one geographical area, and that efficiency would de- 
crease more or less with the distance from that area. As for time, 
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even in the area of maximum efficiency such a test would continue 
to be so efficient only if the same body of facts continued to be 
used, and if there were enough reliable forms of the test to preclude 
the possibility of memorizing and cramming for the test. 

B. Ideas are more nearly universal. A great variety of specific 
information can be used to teach the same ideas. The pupil in 
London, New York, Boston, and San Francisco would acquire a 
fairly large body of common ideas. Thus far, however, only the 
single, less complex ideas have lent themselves readily to the 
new-type form and the corollary statistical treatment. This limits 
the extent of the usefulness of such a test as a measure of advance 
in learning. Again, it has been found that there are many ways of 
expressing the same idea and that different ways are preferred in 
different areas. There would be consequent variation in maximum 
efficiency in different areas. T he same area, moreover, changes its 
preference in terms for the idea from time to time, and this would 
render the test less efficient in any area at later times. It might 
be argued that widened social intercourse renders the standardiza- 
tion of the vocabulary of social ideas increasingly important and 
that, whatever the local preference for variety of expression, this 
should be accompanied by a knowledge of the standard term. This 
form of test, therefore, offers the greatest possibility of standardiza- 
tion. 

C. The other values in social science instruction have not yet 
lent themselves sufficiently to the new-type technique to consider 
the possibility of standardized tests. Psychologists and social 
scientists may ultimately devise such tests, but at this writing too 
much work remains to be done to afford more than a hope. 

VI. THH PROHAHLH LlvXRNING PROCESS IN SOCIAL SCIENCE 

* 

This discussion would be incomplete if it did not consider the 
learning process in the social science field, even if little can be set 
down with assurance at this time.' Psychology has concerned 
itself but little with what it calls the higher thought processes 

‘Sec alstv pp. 14- U>, 45-5 J. 


[ 113 ] 



TESTS AND MEASUREMENTS 


and still less with the higher thought processes in the social 
sciences. Educational measurement, though it has done some 
pioneer work in this field, has as yet advanced little beyond the 
pioneer stage. It is doubtful whether substantial progress will be 
made on this problem until the social scientist and expert in edu- 
cational measurement combine in genuine cooperation. Mean- 
while the following suggestions may be offered as the result of the 
cooperation this Investigation made possible. The.se' suggestions 
are of varying value. Some can be stated almost with the force of 
definite conclusions, while others can be advanced only as conjec- 
tures w'hich some years of teaching experience, observation, and 
attentive study have justified. As such it is hoped that thej'^ may 
stimulate others to more definitive research. They are presented 
with due reservations. 

1. The social sciences are expected by society to have practical 
applications. They are derived from a consideration of social phe- 
nomena and are intended to be of service in the conduct of human 
affairs. Thej'^ are functional subjects, that is, they art* meant for 
use to mankind. They are functional in two senses. They are 
functional in a passive sense when their service is to satisfy general 
curiosity, give added pleasure in helping to explain matters which 
come to the attention, and allay needless fears about the possibili- 
ties of new developments in a dynamic and changing society. 
Whatever they add to the cohesion of society by supplying com- 
mon ideals and traditions may be included as part of their passive 
function. They are functional in an active sense when their ser- 
vice is to inform, modify, or direct conduct. No tests of conduct 
were attempted in this Investigation, though several of the tests 
implied effects on conduct. For the social scientist the thought 
that a knowledge of social science is a conditioning factor in social 
conduct represents its chief functional value, one of the goals of 
his efforts in teaching. 

2. As a matter of conduct, the child begins to learn about society 
almost at birth and continues to learn more or less throughout 
life. Such learning is truly functional, acquired through activity 
and expressed through activity. It is the learning derived from 
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experience with society and is concerned with the direct contact of 
the individual with his social environment, 

3. The rapidity with which the child acquires this knowledge 
and retains it is, in part, dependent upon his ability to identify his 
experiences and to extract from them those elements w'hich are apt 
to be of use to him in the future. The identification of such ele- 
ments by distinctive names is the most economical device which 
society has discovered for this purpose. 

4. The child is aided in this process by the older members of his 
social environment who have had similar experiences and who sup- 
ply names for the essential and functional elements of the experi- 
ence. They may or may not always be at pains to supply him with 
the names most widely used for these elements of experience. 
These more widely used or standard terms the school is expected 
to supply. 

5. Much of the child’s early school days is spent in learning the 
terms which society has found most generally satisfactory' to ex- 
press the functional elements of the experiences which he has 
already had. Pronunciation, spelling, reading, and writing are all 
involved in this process. The acquisition of the names lags far 
behind the experiences themselves in his early school years and is 
still some distance behind at the end of his schooldays. Perhaps 
it would be more correct to say that the gap between experiences 
and the appropriate terms by which to identify them is never 
fully bridged. 

6. Such experience is, in his earlier days, limited almost entirely 
to personal contact and sensory perception. It is incomplete in 
connection with rho.se materials, persons, and acts which come 
within his range propelled by forces beyond the reach of his senses. 
These more remote elements, though part of his own experience, 
he must learn from others who have been able to trace similar ex- 
periences back to their sources, or apparent sources. The service 
of the social science subjects in this connection is obvious. 

7. 4'he accumulation of the functional elements of his own ex- 
perience constitutes the core of his knowledge hearing upon con- 
duct. What he learns about the more remote factors affecting 
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matters which require his action must become connected with this 
core to become functional, that is, to modify his conduct. Social 
science instruction will succeed in its aim of conditioning conduct 
only to the degree to which it succeeds in establishing this connec- 
tion, attaching the more formal learning of the clas.sroom to this 
active core. 

8. There appears in general to be some regular order in which 
the child acquires functional knowledge from experience. He rec- 
ognizes, first, the concrete elements — artifacts, persons, and action 
— and identifies them. I’heir relationship to himself, their relation- 
ship to one another, their relationship to more remote concrete 
objects are appreciated successively, and the identification of this 
relationship follows the same order. The rapidity with which the 
individual child acquires this knowledge depends upon his ability, 
his interest, and his instruction. 

9. His learning about matters more remote, beyond the reach 
of contact, follows a similar order, except that it is preceded by 
words, instead of being follow'ed by words. As a rule he acquires 
first the verbalisms, then the material connotations of the words, 
then the functional ideas. His appreciation of the functional ideas 
depends upon the extent to wdiich he realizes the concrete experi- 
ence from w'hich it is derived and this, in turn, upon the closeness 
with which that concrete experience can be related to his own 
similar experiences. 

10. T he acquisition of functional facts from the experience of 
others is an extended process. Left to himself in reading or hearing 
about the experiences of others, the pupil will gain the concrete 
elements in varying amount according to ability, interest, and 
vividness of presentation. The functional elements of that ex- 
perience will escape him unless especially emphasized. Repetition 
and thought are necessary. 

11. Precision in the grasp of a functional fact requires a number 
of considered illustrations of its occurrence, preferably at intervals. 
Under the most favorable conditions, several illustrations are re- 
quired, and usually many more are needed, to give the pupil an 
understanding of any term as it is commonly used. Should be 
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need this term for active conduct, even more would be required. 
The result of such tests as those of Wesley, Kelty, and Parker and 
Calkins indicated that, even in the case of relatively simple func- 
tional facts, some pupils did not attain until the twelfth grade the 
degree of precision that other pupils had acquired as early as 
the fifth or sixth grades. 

12. Pupils of lower ability can be taught functional facts derived 
from remote experiences, but they require more illustrations, greater 
emphasis, and a longer interval of time than do the brighter pupils. 

13. The effort to teach functional facts by definition and thus to 
hasten the process of imparting the world’s experience is illusory. 
Until their meaning is fortified by experiential material they are 
mere verbalisms, and, as verbalisms, are soon forgotten. The wider 
and deeper the experience of the pupil, the more receptive he will 
be to definitions and principles. 

I'his conclusion suggested by the trial of several of our tests is 
corroborated by the findings of the Modern Foreign Language In- 
vestigation, as also by the Classical Investigation. Their problem 
was concerned with the relation of the rules of grammar to the 
functional use of the language. Both found it a mistake to teach 
formal grammar before the pupil had had experience with the lan- 
guage. The emphasis of the rule in connection with actual instances 
of its occurrence proved more effective. 

14. Organized relationships such as are involved in schemes of 
government, organized industry, social institutions, and highly de- 
veloped activities are compounds of functional facts. At first expo- 
sure, the pupil will grasp only the main nexus of this pattern, which 
will have for him only the value of a simple fact. The whole com- 
plex will not be appreciated until late in his school days, if then. 
Thus a bank is merely a bank, the detailed ramifications and sig- 
nificance of which for the processes of economic activity are only 
learned later. 

15. A system of ideas, like a pattern of relationships, is grasped 
first as a single idea and usually requires some material as.soclation 
to be retained, frequently by imaginative personification. Symbols 
may be required as aids for the beginner. 
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16. A person’s standard of knowledge about any body of organ- 
ized relationship is the amount of knowledge of it which he pos- 
sesses. If he has any knowledge of it at all, he will assume that he 
has complete knowledge, until his knowledge of it is too great to 
be recalled in a moment. 

17. A person forms opinions upon which he is ready to act easily 
in response to the dictates of apparent self-interest and momentary 
expediency. To base such opinions upon knowledge and to qualify 
his action by knowledge is an ideal of whose attainment the indi- 
vidual is most certain in inverse ratio to his actual knowledge. I'he 
more knowledge he has, the less ready he is to act on impulse. 

18. A person usually generalizes his own experience and observa- 
tion, and is not ready to modify that generalization except as addi- 
tional modifying information is forced upon his attention. 

19. A person, left to his own devices, will confine his learning to 
those matters which fall within the range of what he regards as his 
own interest. All other matters will be quickly disposed of under 
some convenient classification as matters of indifference or aversion. 

20. Matters of learning not within the field of direct personal in- 
terest will be regarded as definite, final, and permanent, and the 
possibility of change in such matters of learning will be ignored 
and even resisted. 

21. Interrelationship is appreciated with difficulty. Experiences 
are treated first as separate entities. Their relation to other experi- 
ences are next appreciated as single relationships. An organization, 
as, for instance, an army, is first grasped as a series of single rela- 
tionships, from that of private soldier to commanding general. The 
fact that each further step also involves lateral relationships is not 
appreciated until much later. It is doubtful whether the complex of 
relationship represented by an institution is clearly grasped before 
the end of the high school. The appreciation of a complex of ideas 
behind the institution is even more difficult. It can be only partial 
at the end of school days. 

It is advisable to state again what was said at the beginning of 
this discussion — the work of test construction was directed chiefly 
toward those matters which the social sciences had in common, 

(1181 



THE PROBABLE LEARNING PROCESS IN SOCIAL SCIENCE 


those to which each contnhuted more or less directly. The techni- 
cal know'ledge and skill which each of them develop separately have 
been for the most part disregarded. This arbitrary procedure was 
justified by the fact that those separate contributions have been 
most fully emphasized in the vast literature on this problem. The 
value of those separate contributions is not affected by this em- 
phasis upon the common elements. The skills, the acquaintance 
with potential sources of information, the technical knowledge and 
the great reserve of information and ideas which each has to offer 
are probably only to be acquired in the process of dealing with the 
materials of each separate field. 

One of the conclusions most forcibly gained from this study is 
that the study of the problem of learning in this field is only just 
begun. Despite all the effort which has been devoted to the task 
not only by this Investigation but by the many w-ho have attacked 
the problem before, we are scarcely over the threshold of the diffi- 
culty. Enough has been learned to indicate that the solution of the 
problem, or, indeed, any real advance toward the solution, will re- 
quire the best efforts of the social scientists themselves as well as of 
the scholars in educational measurement and applied psychology. 
Unless this united attack is made, little real advance will occur, for 
in the very nature of things the expert in educational psychology 
can not also advance to expert knowledge in the social sciences; and 
without that he cannot hope to advance beyond the elementary 
stages of the problem. The sympathetic and intelligent cooperation 
of the social scientist will be essential to advance the problem 
beyond that stage. 
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Chapter II 

PLAN AND MEl'HOD OF TEST CX)NSTRUCT10N 

The classroom teacher knows that there is a wide difference be- 
tween wrhat may be offered to the pupil and what that pupil re- 
ceives or takes away. Parents who concern themselves with the 
instruction of their children also realize the truth of this statement. 
Those accustomed to dealing only with adults often forget this 
fact. An investigation concerned with an educational program for 
the elementary and secondary school must take it seriously into 
account. 

But the task of discovering how the pupil builds up his knowl- 
edge of society is by no means an easy one. The only source of 
that information is the pupil himself. His testimony alone is con- 
clusive. To obtain his testimony in an efficient way that will yield 
data useful to those engaged in the construction of curricula and in 
school instruction presents difficulties. Some form of test through 
which the pupil will reveal his hazy approach must be employed. 
But what form of test? 

When the Committee which planned this Investigation began 
its work, faith in the efficacy of the new-type technique for educa- 
tional measurement was at a very high point. 1 his instrument, 
which had been developed by psychologists, had achieved wide- 
spread attention in the measurement of general intelligence and 
had been extended with almost equal success to educational meas- 
urement. The w'ork of Rice, Ayres, Thorndike, and their followers 
in the measurement of elementary school subjects had become al- 
most universally accepted,' lmj>ortant progre.ss was being made 
in the measurement of more advanced subjects of high .school and 
even college grades. The Modern Foreign Language Study, which 
was just completing its w'ork, had been peculiarly fortunate in its 

’It is impossible here to accord due credit to the many scholars who, as pro- 
fessors of psychology or of educational research, have made significant amtrihutions 
in the measurement of instruction. The list of those devoting study to specific 
school subjects includes such workers as Buckingham, Courtis, Dcatborn, Gates, 
C. T. and W. S. Gray, Haggerty, Horn, Judd, Irabue and Van Wagenen as well 
as many others. 
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experience with this new instrument. With the assistance of Hen- 
mon, Wood, and Buswell, tests had been constructed to measure 
progress in the knowledge of vocabulary, grammar, and reading. 
Specialists in modern foreign language teaching had been skepti- 
cal, if not actually hostile, to the use of the device, but those most 
intimately associated with the investigation were enthusiastic in 
their praise of the results obtained through the use of the tests. 

In view' of the great advance that had been made in educational 
measurement and the high endorsement given to that work by the 
subject matter specialists in modern foreign languages, the com- 
mittee planning the Investigation of the Social Studies was led to 
hope that this instrument might also prove very useful in their 
work. As yet, few of the subject matter specialists had taken any 
great part in the construction of such tests. The chief burden for 
the development of the tests still rested upon the psychologists. 
1 he latter had been required to do nearly all of the w'ork on tests 
designed to measure educational advance at the secondary and 
even college levels. In exceptional cases, such as that ofHenmon, 
who was himself a highly accomplished scholar in modern foreign 
languages, and of Wood, who studied law in preparation for con- 
structing the tests for the Columbia I. aw School, the tests repre- 
sented a combination of knovslcdge of subject matter as well as of 
psychological technique. In general, however, the work of test- 
construction was still largely in the hands of psychologists whose 
knowledge of the special subject matter fields w as limited. Whether 
the higher degree of objectivity resulting from a lack of special 
knowledge would be sufficient to overcome the deficiences in that 
knowledge remained still to be determined. Though there was 
some skepticism among them, nevertheless the planning committee 
felt justified in prov iding for a somewhat extensive testing program. 

It was the hope of this committee that its testing program might 
yield instruments by which the relative effectiveness of various cur- 
ricula and various teaching procedures in gaining desired results 
might be determined. The great activity in curriculum construc- 
tion since 1916 had led to the development of many new' or differ- 
ent arrangements of courses, subjects, and even materials in the 
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"field of social studies.* While the greatest amount of experimenta- 
tion had been centered on the three grades of the new adminis- 
trative unit, the junior high school, there was enough of such ex- 
perimentation in the elementary, senior high, and junior college 
grades to afford profitable comparative material. Should the re- 
sults of the test program reveal decisive advantages in any parti- 
cular program, or method of procedure, or in both, such results 
would afford a sounder basis for any recommendations the Com- 
mission might make. 


’a co.mposite of some social studies offered in 
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Determination of Obiectives for Test Purposes 

The Committee realized, however, that hope and the achieve- 
ment of the task were not identical. It recognized some of the dif- 
ficulties in the way quite clearly before the work was begun. The 
Investigation involved a number of subjects which were commonly 
organized in separate academic departments. A relationship be- 
tween these several fields of specialization was recognized, but the 
scholars in all of these fields had been so intensively occupied with 
their own special fields that few of them had had time to consider 
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Courses Offered in the High Schools in 1924 




DETERMINATION OF OBJECTIVES FOR TEST PURPOSES 

Local History 

Mythology 

Banking 

Business Management 
Foreign Relations 

the exact nature of that relationship. For the sake of economy, 
moreover, it was desirable to narrow the work of testing to those 
aspects of the social sciences which offered the maximum amount 
of comparison. An effort was therefore made to state objectives or 
aims which alt members of the Committee could accept as desir- 
able outcomes ofinstruction in the social sciences. Such a state- 
ment might serve as a guide to the work of test construction. 

It proved to be no easy matter to arrive at ^ch a statement of 
objectives or aims acceptable to all of the Committee. It was dif- 
ficult to state these aims in terms of specific subject matter because 
such a statement might limit comparison too narrowly to particu- 
lar subjects or to schools which offered the same subjects in the 
same grades. While valuable, this procedure would multiply the 
number of tests beyond any possibility of completion. Further- 
more, the ordinary procedure in schools involved frequent tests of 
the learning of separate subject matter, whereas the Commission, 
because of its wide membership, was peculiarly well equipped to 
consider the aims and values which the social sciences had in com- 
mon. Accordingly, it seemed best to emphasize, for the guidance 
of test construction, those objectives, aims, and values which the 
social sciences had in common. This involved a statement in gen- 
eral terms and one which was intended to have validity in all of 
the social science fields. 

After extended thought and discussion, the Committee was able 
to agree tentatively upon such a list of objectives for guidance in 
test construction. This list was included in its working drawings 
presented in 1928. 

1. Understanding of important institutions by means of which society functions. 
Principles and ideals are included. 

This is the most generally accepted function of the social studies, and the 
most precise. Understanding and information are not rynonymmis; and testa 
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Chart Drawn for History and Social Studies Investigation November, 1929 — Truman L. Kelley 
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principles considered important by three members of the Commis> 
sion, E. B. Greene, Henry Johnson, and the Director. A test or 
sub-test could conceivably be constructed for each cell in the Table. 
( + * test.) 
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It is here intended to measure the pupil’s time sense, his 
jibce settac, the two conjointly, and ajso his sense of continuity of related events 
whtdt, in addition to sense of continuity in time and place, calls for an appreda- 
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tkm of the continuior of cultures, disdplinee, of inititutioiii and of thousht, ai, 
for example, is involved in the understanding of the Roman Catholic church or 
the continuum representing the development of painting. 

t2 — Histoned Setut: It it here intended to measure the thoroughneu srith 
which the individual transposes his mental outlook to the time, group, or person 
historically described; his ability to see a century through the eyes of that period; 
to grasp a motive through the outlook of the one whose act it involved; and to 
judge a character with such understapding as the character itself, as well as the 
observer, may experience. 

t3 — Sense of Evidejue: It is here intended to measure the critical skill inth 
which the pupil appraises a bit of evidence coming to him under various conditions 
and with various defects; to measure the soundness with which he draws deduc* 
tions and makes generalizations; and to measure the mental standards which he 
maintains in judging of evidence. 

t4 — Interpretation of Present: This in one sense includes all the other values 
of history. It is, however, intended to measure here such aspects as are not coveted 
by tl, t2, and t3. Here are to be investigated specifically acts of interpretation 
of happenings in the lives of the pupils and specific personal modifications in conduct 
or intellectual outlook. 

tS — Interrelcuionskips: Whereas tl, t2, t3, and t4 have treated of history as 
an individual discipline, we here wish to examine such relationships as have been 
established in the pupil’s mind between history and other social studies and 
activities and other sciences. Personal values and independent points of view 
involving history and several ocher phases of life at the same time may be here 
investigated. 

The Commission approved the principles and program presented 
by Dr. Kelley and appointed an Advisory Committee on Tests to 
aid in the execution of this work. The Committee consisted of 
Messrs. F. W. Ballou, Chairman, I. Bowman, H. C. Hill, Ernest 
Horn, Henry Johnson, A. C. Krey, and Ben D. Wood. In consulta- 
tion with the Committee projects were assigned to individuals and 
active work was begun. 

The Selection of Items Composing the Tests 
IN THE Social Studies 

Immediately preceding,^ a general outline is given of the field 
covered by the social studies in the schools. This is the region 
wherein it was desired to construct tests. The essential service 
which it was hoped to render by the measuring devices developed 
was to enable investigators to ascertain whether the values sought 
and measured by the tests employed were being attained by a given 
regimen of instruction. For example, if it be granted that the pur- 
pose of instruction in the social studies is to develop good citizens, 
*See p. 122 ff. 
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nevertheless this idea is serviceable in connection with school prob- 
lemsonly to the d^ree that it becomes so delimited that some means 
(judgments of teachers or associates, or tests) gives information as 
to whether, and to what extent, pupils give evidence of “good citi- 
zenship." In so far as good citizenship is not quantitatively or 
qualitatively measurable by tests or the personal evaluation of 
associates, it is worse than no concept at all, as it distracts the at- 
tention from concern with attainable and cognizable outcomes. 
The moment it is possible reliably to appraise pupils upon this 
trait, it becomes possible to determine the efficacy of instruction as 
stimulating the trait; it becomes possible to choose between several 
types of instruction, or types of curricula, upon the basis of their 
excellence as developers of the trait. 

The technique for determining the efficacy of a procedure is well 
established. It is that of the controlled experiment wherein the out- 
comes in terms of the improvement of pupils of two groups, as simi- 
lar as possible in every respect except the one being investigated, 
is determined. This technique will not here be elaborated, but it 
should be borne in mind that it alone provides the setting wherein 
objective measurements of pupils can establish their serviceability. 
The basic purpose of the test construction work of the Test Com- 
mittee has been to make possible the determination of the rate of 
growth of a class under a certain tutelage and the comparison of 
this rate with that under an alternative environment. Unfortu- 
nately the time required to construct tests, and other limitations of 
the work of this Committee, have not permitted thorough-going 
investigations of alternative types of instruction in the social stud- 
ies by means of carefully controlled experiments employing the 
tests developed by the Committee and other tests. These things 
remain to be done in the future, but it is believed that a number of 
measuring instruments serviceable for this purpose are here made 
available. 

The general field of the social studies as indicated in the preced- 
t(tg sectimn must be broken up into subsidiary fields as there indi- 
cated* and these into still \more subsidiary, before a point is reached 
fdienecR specific instruction can function and wherein specific 
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measurement can be brought to bear. This process of analysis and 
subdivision is distinctly the province of competent social scientists, 
exercising judgments based upon the sum total of their experiences. 

The analyses made by such authorities differ, and the issue that 
confronts the maker of a soeial studies curriculum, or a social 
studies test, is how to proceed in the face of these differences. The 
simplest way out would be to disregard the authorities and build 
upon tradition. The accompanying table may help to make clear 
the nature of the issue. 


SPONSORSHIP 



P.T. = Pioneer Thinkers 

T.A. = Traditional Authority 

Way Utilized 

The problem of first im- 
portance here is how to 
select these authorities. 
Having them, the prob- 
lem of how to get and 
utilize their judgments 
becomes very important. 

A sampling of present prac- 
tices in regard to social studies 
curricula, and of the present 
needs of a citizen to apprehend 
language, written or spoken, 
etc., defines this traditional 
authority. What constitutes a 
fair sampling is the point of 
importance here. 

Purpose 

and 

Content 

Outcomes 

of 

Each 

Approach 

Nuclear 

P.T.N.=Pioneer Think- 
ers Nuclear outcome. A 
small nuclear consensus. 

T..A.N. = Traditional Author- 
ity Nuclear outcome. A large 
nucleus represented by current 
practice. 

Divergent 

P.T.D.—Pioneer Think- 
ers Divergent outcomes. 
Wide individual differ- 
ences as to purpose and 
content. 

T.A.D. =Traditional Author- 
ity Divergent outcomes Oc- 
casional variations, each the 
outgrowth of some fraction of 
tradition. 


It is conceivable that the objectives discovered from either the 


P.T. or the T.A. approach might almost completely overlap each 
other, but hardly conceivable that the emphasis would be similar. 
Pioneer Thinkers tend to concern themselves with those matters 
in the social scheme which require modification and, further, to 
emphasize such matters in order to gain the desired attention. It ' 
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is to be expected that any seleedon of objectives based upon (Honeer 
thinkers alone would result in incomplete and distorted views and 
values, just as it is to be expected that any determination limited 
by traditional authority would fail adequately to represent society 
in its growing aspects. 

We can readily find fault with each of the four sorts (P.T.N., 
P.T.D., T.A.N., T.A.D.) of resulting material if used alone for the 
determination of curricula and of measuring instruments. In brief: 

Traditional Authority Nuclear Outcome: The nuclear material 
of tradition is static and insufficient in a changing social order. 

Traditional Authority Divergent Outcomes: The divergent pur- 
poses and practices bequeathed by the past may be so numerous 
as to require selection before they can serve in an actual measure- 
ment or teaching situation. In this case the problem of selection 
calls for ''authorities’* and the situation becomes indistinguishable 
from one admittedly based upon the convictions of experts. 

Pioneer Thinkers Nuclear Outcome: The nucleus found as a con- 
sensus of "best minds" is automatically made smaller and smaller 
in amount as the number of authorities is increased. It thus be- 
comes most inadequate when the richness represented by the au- 
thorities is most complete. Even with a dozen or more authorities, 
the substance of a consensus may become almost trivial in com- 
parison with the richness of thought of any one of the separate 
authorities. This consensus may be considered satisfactory as far 
as it goes, but should not circumscribe curricula or measurement 
efforts. 

Pioneer Thinkers Divergent Outcomes: The divergent views of 
pioneer thinkers automatically become greater and greater in num- 
ber as the number of authorities is increased. This approach leads 
to an unwieldy mass of material. Within this material, however, 
may be expected to lie the pa,ths of real progress. 
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If we now endeavor to allocate Test Committee projects to some 
one of the four sponsoring fields mentioned, our result would be as 
follo\ra: 

Traditional Authority Nuclear Outcome: Measurement of un- 
derstanding of social science terms projects by Edgar B. Wesley, 
Mary Kelty and Nelle £. Moore, Mary Gold, and Luella Cole 
Pressey.^ Also measurement'bf informational aspects of history as 
initiated, but not completed, due to Committee limitations, by 
David P. Harry, Olive Bucks, and Janet Cameron. 

Traditional Authority Divergent Outcome: Nothing. 

Pioneer Thinkers Nuclear Outcome: Measurement of social at- 
titudes toward specific features of society as carried out by L. L. 
Thurstone. The measurement of character traits as undertaken by 
Truman L. Kelley, M. R. Trabue, and A. M. Jordan.* (Perhaps a 
minor portion of the above should be classified under P.T.D.) 
Measurement of fundamental concepts to be derived from a study 
of geography (part of test material devised by R. D. Calkins and 
Edith Parker).* Measurement of certain general outcomes of in- 
struction in history (part of test material devised by Marion Gark).* 

Pioneer Thinkers Divergent Outcome: Measurement of geog- 
raphy concepts (part of test material devised by Calkins and Par- 
ker). Measurement of general outcomes of instruction in history 
(part of test material devised by Marion Gark). 

It will be obvious that such tests as have been devised are en- 
tirely inadequate to represent the divergent views of careful stu- 
dents of the social studies (P.T.D.), wherein one may expect to 
find, along with much abortive material, the real suggestions for 
social progress. Of the tests developed by the Committee the one 
which is most inadequate as a measure of tradition is the Calkins- 
Parker Geography Test, or at least the major portion of it, while 
it is at the same time the most promising one developed as a meas- 
ure of what may prove to be an important and real line of develop- 
ment in future work in geography. 

'See Chap. III. *See Chap, V. *See Chap. IV. 
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A quite radically different technique is appropriate in the con- 
struction of tests which are the outgrowth of sampling traditional 
practices (T.A.) from that appropriate to tests sampling the ideas 
of pioneer thinkers (P.T.)- When tradition or current practice con- 
stitutes the warrant for the inclusion of material in the curriculum, 
it is sufficient to ascertain what current usage is, and construct 
test items in harmony therewith. The resulting test scores meas- 
ure the extent to which a child is aware of the prosaic contempo- 
rary social life about him. If a word such as “financier” is in com- 
mon usage in the social studies and in newspapers and magazines 
of wide circulation, it calls for the judgment of no social scientist 
of renown to establish that knowledge of the word is to a degree 
(of course slight, as but one word is in question) indicative of a 
higher level of understanding than lack of knowledge of it, and 
that such knowledge is in fact “good.” In the dim past some pio- 
r»eer thinker introduced the term, but that is long forgotten and 
today it is merely a matter of noting that the word serves in un- 
derstanding life about us to conclude that development of knowl- 
edge of it is a desirable feature of a curriculum, and that measure- 
ment of understanding of the word is an informative service that 
can be rendered by a test. Clearly all common concepts are of this 
nature and this is irrespective of any emotional tone accompany- 
ing them. The arch-conservative could hardly deny that to know 
the meaning of the now common word “communist” indicates a 
higher level of social understanding than not to know it, and thus 
“communist” becomes a fit word for inclusion in a social concepts 
test irrespective of sponsorship by eminent social scientists. 

With these introductory remarks we may now proceed to a state- 
ment of the specific steps to be followed in constructing a test of 
knowledge of terms used in the social studies. The selection of 
terms to be incorporated into the test comes first. These terms 
are to be drawn from a very great number — if possible, from a 
number so great as to represent practically the entire field. The 
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8te{>s actually followed in obtaining such an initial list have been 
described in connection with the various tests developed to meas- 
ure knowledge of social terms. The list of social terms is so long 
that it obviously is not feasible to incorporate all of them into the 
tests devised. A sampling of these terms is resorted to» and is de- 
pendent upon the judgment of the person actually constructing 
test exercises. Because this person must exercise judgment in the 
selection from the larger list of the words from which he will con- 
struct test exercises, and because his judgment is involved in the 
actual construction of each test item, his work, but not that of 
those making the initial word counts, is to be subjected to the 
scrutiny of other students of the social sciences. 

The form of test item that he employs may be investigated ex- 
perimentally by trying a number of forms and choosing the one or 
two yielding the most reliable results (as was done in the case of 
the Gold tests), or it may be decided upon from general considera- 
tions built up from a knowledge of current tests in related fields 
(as in the main was the case with the other tests developed of so- 
cial terms). The result of both of these approaches was an adop- 
tion of the multiple-choice test with four or five options. 

The initial efforts at item construction were scrutinized by the 
psychologist of the Commission for errors in form. Things that 
were especially noted were: 

(a) Accuracy as determined by reference to a standard diction- 
ary, not as judged by a teacher of the social sciences. 

(b) Fitness of balance between the word difficulty of the essen- 
tial word being tested for and the words ancillary to the process of 
testing. For example, to employ the word “indescribable” (having 
a frequency value of 5, i. e., in the eighth thousand in the Thorn- 
dike Word Book) when testing for knowledge of “church” (fre- 
quency value of 81, i. e., in the fifth hundred in the Thorndike 
Word Book) would pervert the test, it becoming a test of knowledge 
of “indescribable” and not of “church.” 
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(c) Lack of lecondary responce cues. For example, if the phrase 
representing the correct response option is generally longer than 
those representing the alternative options, then the length of option 
becomes a cue to response. There are many secondary cues: an 
order of response option other than chance may be one; special 
type of qualification of option, as, for example, those employing the 
words “never," “always," “greatest," “least," and ultimates of any 
sort, may commonly characterize incorrect responses. Grammati- 
cal inconsistency may be such a cue, for ordinarily the correct op- 
tion is euphonically and grammatically in agreement with the ini- 
tial portion of the exercise, but “fillers” merely stuck into to pro- 
vide options are less certain to be. 

(d) The order of difficulty of test items in preliminary forms of 
the test. It is a dogma of good test practice, easily defended, that 
the items of a test should be such, and so arranged, that the test 
starts with the easier items and continues by as regular steps as 
possible to the harder items. 

(e) Fairness with which the field was sampled. Though this 
point was at times considered by the psychologist, it was mainly 
left to the test author and the social science critics. 

The things mentioned all occur early in the process of item scru- 
tiny. They were followed by a review of items by social science 
specialists to insure essential accuracy in the technical aspects of 
the subject. 

I would here mention some failures in procedure. This is done at 
the risk of seeming to be unsympathetic with the trials of the social 
science critics, but it is in truth done with the view to pointing out 
the serious difficulties confronting one who attempts to use high 
grade talent in criticizing elementary school items. Most words 
possess a hierarchy of meanings, as, for example, the word “rail- 
road." B^inmng with the local railroad station and train as evi- 
dence of primitive meaning, one may continue to systems, mergers, 
Interstate Commerce Comniission, as elements in a fairly full mean- 
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ing. For a child to get credit for saying that a railroad consists of 
tracks with cars upon them, may be considered ludicrous by the 
university professor of transportation. However, this may repre- 
sent a degree of knowledge quite accurately characterizing first 
grade understanding, and not kindergarten understanding. In other 
words, the item may have a legitimate function as a primary 
test item. The authorities, generally university men, who so kindly 
cooperated with the Commission in judging items, proved to be 
very severe critics, tending to eliminate certain very sound items 
merely because of a difference in the conceptual level in the mind 
of the deviser of the item from that in the mind of the critic. 

Further, an allied outcome of the judgments of items was the 
general severity of destructive criticism unrelieved by constructive 
suggestions. Had it been required that each adverse criticism be 
accompanied by a constructive modification of the item criticized, 
the work of the devisers of test items would have been greatly sim- 
plified. The observation that an item is inadequate, for some very 
general reason such as that *'it misses the point,** is not especially 
helpful. Under the conditions existing, which were that the judges 
donated their services, to ask for constructive criticisms would have 
been unreasonable, but it would not be unreasonable if services 
were adequately paid for. The point of this discussion is in no 
sense to find fault with the judges of the tests or with the manner 
in which they worked. It is rather to show the inherent difficulties 
involved, in order that future endeavors may not be mired or made 
difficult by the same problems. The psychologist cannot but feel 
the highest esteem for certain authorities, whose lives are dedicated 
to college and graduate school instruction, for the consideration 
they showed when called upon to judge items designed to measure 
elementary and high school levels of ability. The judgments ren- 
dered have resulted in an improvement in the elementary and high 
school tests, but at the cost of annoyance to the judge and to the 
deviser of the test items (for all complaints ultimately reach himj, 
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that it is desirable to avoid. The essential suggestion offered to 
avoid the difficulty noted is that judges be adequately paid and 
asked to accompany each adverse criticism with a constructive sug- 
gestion. It is not suggested that judgments of experts of profes- 
sorial grade are not needed and should not be called for in connec- 
tion with the construction of elementary test items in the held of 
the social studies. 

Having received the criticisms of the judges, the author of the 
test then revised the items, and the process of criticism was re- 
peated with the idea that finally a test would be constructed that 
would satisfy both the author and his critics. Generally speaking, 
this was accomplished, though it should be noted that there was 
more or less reservation, though not positive disapproval, on the 
part of the judges as to the value and fitness of a number of the 
test items, even of some of those in the final forms. 

TThe items were then arranged, as nearly as possible, on the basis 
of the author's judgment, in order of difficulty and built into pre- 
liminary forms for try-out. These were given under what were in- 
tended to be standard directions to the appropriate grade groups 
dispersed as widely throughout the country as was feasible. All 
papers were sent to the psychologist for scoring and analytical 
study of the performance of test items (except in the case of Dr. 
Pressey's test, which was scored under her direction). 

Information yielded by the try-out. By behavior of an item is 
meant the change in the percentages of correct responses to the 
item from groups of various levels of ability. It is thus necessary 
to have groups of different levels of ability in the function which 
is being tested for. If a test of “introspection” is being devised, 
and a test item thought to measure it (perhaps actually measuring 
it) is given to various subjects, unless there is an independent 
means of segregating the subjects on the basis of “introspection,” 
time is no way of usefully describing the behavior of the item, 
riiat 40 per cent of group A pass, and 60 per cent of group 
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B do SO, means little if it is unknown whether or not group B it 
superior in ^'introspection” to gioup A. 

Three ways of determining groups differing in ability with re> 
spect to the function being measured have been followed and an 
mentioned in order of decreasing merit. 

(a) Judgments of teachers, pr of other associates, have been util- 
ized to select subjects (pupils) having different abilities. 

(b) Age or school grade location has been taken as a basis for 
selecting groups, on the assumption that the higher the grade, the 
higher the ability. This obviously will work well for traditional 
school values and purposes, for here grade location has been de- 
termined by judgments of teachers wholly or in part upon the 
ability of pupils in the mental function in question. This method 
has been largely followed in connection with the study of know- 
ledge of social science terms where increase in age or school 
grade may be assumed at least to go with increased knowledge. 
This method was used, though not with complete satisfaction, 
undoubtedly because of the nature of the functions being tested, 
in studying the Marion Clark historical ability items, and the 
Calkins-Parker Geography Test items. The method was not even 
investigated in connection with the Kelley-Trabue character 
measures or the Thurstone social attitudes scales, for in these cases 
the assumption that pupils in higher grades stand higher in the 
functions involved is clearly not established. 

(c) The performance of pupils upon the entire preliminary test 
has been taken as giving an order of ability in the function which 
could be utilized in studying the performance of the separate items 
successively. 

In a very important way this last procedure begs the question. 
Strictly speaking, all it can do is to reveal whether the single item 
measures the same thing as the average of all the preliminary test 
items. If the large number of items composing the preliminary test 
do, in fact, measure the function they are claimed to measure, ex* 
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cept for chance errors, then the average of them all (characterized, 
at an average it, by greater refiabtlity than the separate measures 
entering into it) does constitute a sound criterion whereby to judge 
the separate items. The method dierefore does have the merit of 
lessening the role of the chance factors entering into the instrument 
of measurement. It has no merit whatever in eliminating system- 
atic errors represented by the difference between the average func- 
tion measured by the items of the preliminary form and the concept 
as warranted by the title of the test. When this third method of 
studying the performance of items is resorted to, the full weight of 
responsibility for the validity of the test rests upon the authors, 
for this validity is in no wise affected by the results of the try-out. 
In the Calkins-Parker Geography Test, for instance, the try-out 
quite generally failed to reveal important grade differences. Ac- 
cordingly, neglecting grade grouping (supposed at the beginning of 
the investigation to indicate differences in levels of geography abil- 
ity) we have left evidence of validity, which goes back to the 
authors of the test items, but not back to differences of performance 
of pupils known to stand high and others known to stand low in 
the ability in question. 

Though study of the performance of an item in groups scoring 
successively higher upon the total preliminary test provides no 
basis for keeping or rejecting the item, becaus^ it does not measure 
the function called for in the label of the test, the performance of 
an item does provide a very good basis for selection for the excel- 
lence with which an item measures whatever it is that the average 
of the remaining items in the tiy-out form measures. Items selected 
in this manner can be built into a final test which does consistently 
measure something, but whether it is what the test label calls for 
remains a matter of judgment. 

In the case of material for which it was assumed there would be 
progress from school grade to grade, item profiles were constructed, 
as, for estamiAe, the fcdlowing taken from the Kelty-Moore Test of 
Gsilcepts in the Sodal Suidies: 
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Item A: The teims of a treaty are 
the length of the treaty 
the matters agreed upon in the treaty 
each half of the treaty 
how long it took to make the treaty 
the places where thf names are signed 

Item A. Profile of a Good Item 

Per cent Grades 


correct 

4 

5 

6 

7 

8 

9 

100 

• • • 

• a 

a a 

a a 

a a 

‘98 (9a) 

90 

« • a 

• « 

a a 

■ a 

4 a 

91 

80 

• • • 

4 * 

a a 

82 

85 

84 

70 

« • « 

• • 

a a 

• a 

1 « 

a a 

60 

• • • 


56 

a a 


a * 

50 

• a • 

* * 

•• 

■ a 


a a 

40 4 *••••«■ 

• • • 

• * 

•• 

a a 

•• 

a a 

30 

30 (4a) 

27 

a a 

a a 


a a 

20 

. 20 

• t 

• • 

a a 

a a 

a a 

10 

10 (4b) 

• • 

• • 

• a 

a a 

a a 


0 


• • 


* • 
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Item B: Staples are 

things which always remain steady 

places where horses are kept 

the principal products used in trade and commerce 

trees from which sjrrup is obtained 

posts in the ground 

Item B. Profile of a Poor Item 

Per cent Grades 

conect 4 5 6 7 8 9 

100 ... 


80 

70, 


60 


SO 


40 

30 


38 '(9a} 
33 

28 (9b) 


20 

14 (4t) 

10 

9 

4^ (4b) 

0 


16 


19 


10 


11 
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Item C: To subdue means 
to stop suddenly 
to conquer by force 
to pay money you owe 
overdue 

to give in to a greater power 
Item C. Profile of a Useless Item 
Per cent Grades 

correct 4 S 6 7 8 9 

100 «■ •• ■■ 

90 


70 

60 

so 


40 

30 

20 

10 10 (4a) 

9 

8 (4b) 

0 


25 


25 


19 


28 (9a) 
24 

20 (9b) 


II 
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An item such as has every indication of being a good mea8ur> 
ing instrument. It is good in the sense of reliably measuring growth 
from grade to grade in whatever it is that is involved. One final 
check should be made to provide evidence as to whether the exer- 
cise continues to be a measuring instrument below and above the 
mean 4th grade and 9th grade levels. To be serviceable, the item 
should differentiate for these two terminal classes as well as for the 
intermediate classes between pupils low and high in the ability 
being measured. We may therefore divide the 4th grade into 
two sub-groups, those below the median upon the total preliminary 
test score and those above it. To be a good item at the 4th 
grade level the first of these groups should do less well than the 
second. Similarly for each other grade, but it is hardly necessary 
to go to the trouble of investigation for the intermediate classes 
since evidence is available for grades upon either side. Accordingly 
we will do this for the two extreme classes only, the 4th and 
the 9th. The records of the lower and upper halves of the 
4th and 9th grade pupils are indicated on the profile charts 
opposite the symbols (4b), (4a), (9b), and (9a). Since (4b) is de- 
cidedly below (4a), and (9b) below (9a), and since from the mean 
4th grade to the [mean 9th grade level inclusive there is defi- 
nite and substantial progress in the percentage of correct scores, 
the item meets every statistical requirement of a good item. The 
study here suggested of the performance of the upper and lower 
halves of terminal classes is to be recommended, but it was not 
generally pursued in the actual study of the items in the social 
studies tests, so that the tests as built up are probably not quite 
as reliable for terminal classes as for intermediate classes. 

Item A, having earlier met the requirements of the judges of 
the items as to substantial accuracy, and being known from word 
counts to involve a concept commonly entering into the social 
studies, may be considered entirely satisfactory for measurement 
purposes. 

Items B and C are not satisfactory from the standpoint of their 
behavior. The study of the performances of items has the purpose 
of building up a test from such items as rather than from such 
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as B and C. Items of the A sort will be referred to as *'good’* 
itemSf of the B sort as “poor** items, or as items having little diag- 
nostic power, and items of the C sort as “unsatisfactory” items, or 
items having no diagnostic pov^r, meaning thereby no power to 
diagnose any ability of pupils as represented by differences in the 
mean accomplishments of successive school grades. 

■'i 

Grade Reliability Criterion 

In case there is little or no progress in the function in question 
from grade to grade (as may be the case in, say, cooperativeness), 
but real differences between pupils within any grade, the profile of 
the item would be as shown in D, where (5 b), (Sa),etc.refer to lower 
and upper halves of the classes in question on the total score of the 
preliminary test in question. 


Item D. Profile of a Reliable Item Which is Useless in Diagnosing 
Grade Diferences. (Hypothetical) 


Per cent 



Gradei 



correct 

4 

S 

6 

7 

8 

9 

100 


• • 

•• 

• • 

•• 

• • 

90 

1 • « • 

• • 

•• 

• • 

• • 

% % 

80 

. . 

• • 

• • 

71 (7a) 

• • 

• • 

70 

.. 70 (4a) 

68 (5a) 

69 '(6a) 

70 (8a) 

70 (9a) 

60 

. . 

• • 

SI 

■ • 

• • 

51 

SO 

50 

49 

•• 

50 

49 

« • 

40 

. . 

• • 

33 '(6b) 

' • 

• • 

• • 

32 (9b) 

30 

.. 30 (4b) 

30 (Sb) 

• • 

29 ‘(7b) 

28 (8b) 

« • 

20 

. . 

•• 

• • 

■ • 

• • 

• • 

10 





« • 



0 

[ 147 ] 



TE$TB AND MEAflUAEMEMTa 


Here ia an item which definitely distinguishes between the high 
and the low in the function in question. It shows no systematic 
grade differences, presumably because the function in question 
is not growing from grade to grade. The statistical technique for 
studying the items entering into such a function is thus, of course, 
quite different from that for studying the sort previously men- 
tioned. For studying this function we need to know the reliability 
of the item, or of a sampling of similar items, for each grade. In 
the case of the Calkins-Parker Geography Test, the first endeavor 
was to secure evidence as to good items by studying their grade 
profiles. These very generally showed little progress from grade to 
grade, so it was thought that the items and the function involved 
might be of the Z> sort. Accordingly the reliability of the items was 
investigated by correlating the score on a certain number in one 
category of the test with an equal number of the same category. 
However, reliability appreciably above zero was not found for 
many of the geography items which had been most carefully de- 
vised to measure certain important geographical abilities. With 
reference to these we are thus forced to conclude that the items are 
not measuring anything as yet existing in definite and different 
amounts in the school children of the classes studied. 

A number of these items, however, did show reliability with 
groups of college freshmen, so that evidence is at hand that they 
are measuring something, provided that something exists in the 
pupils examined. We may think of these items as probably being 
ready to measure junior and senior high school pupils in a certain 
sort of geographical ability as soon as such pupils have been suffi- 
ciently exposed to the ideas inherent in this ability to show and 
develop individual differences. The test has thus not established 
itself as a serviceable measuring instrument, but for those geogra- 
phers who trust the ability of the sponsors of this test to define 
important geographical concepts it is a challenge so to instruct 
pupils that the ability in question does come into existence. This 
ia a very impoKtant challenge and it has been brought to light by 
the attempts to measure an important function. This fact should 
be noted by sqch critics of test work as maintain that no basis for 
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''progress” is to be found in the measurement movement, (hi the 
contrary, the movement teems with many suggestions to one in* 
terested in measuring intellect in whatever manner it may be con* 
ceived or may show itself, rather than in measuring some predefined 
and circumscribed ability. 

Validity Criterion 

We may consider increase in percentage of correct responses 
from lower to higher school grades as evidence of some ability 
which does progress from grade to grade, but it is not evidence as 
to what this thing is, whether, for example, it is spelling ability, 
weight, computation ability, or something else. Having an item 
yielding the profile we are not^sure whether it is a spelling item, 
a measurement of weight, or what not. What it is has been assured 
to us only by the judgments of the author of this test and his ad- 
visers. Now it sometimes happens that such judgments may be 
very inaccurate. For example, an item intended to measure honesty 
might in fact measure dishonesty; that is, a high score anticipated 
to be indicative of honesty might be indicative of the opposite, just 
because the deviser of the item was not sufficiently expert to ap- 
praise correctly all the psychological elements determining the 
ways and the reasons of a pupil answering the item. In this case 
it is quite unsound to construct test items on the basis of judg- 
ments. (An example rapidly becoming classic is with reference to 
introversion-extroversion where substantially as many different 
sorts of functions have been claimed to be introversion-extroversion 
as the number of tests of it built up by different authors.) Here 
the appropriate procedure is for the judges to select subjects 
(pupils) standing low, and others high, in the trait, but not to 
select or construct test items, and then experimentally discover 
test items which segregate the subjects in the same manner as they 
have been segregated by the judges. Such items, no matter what 
their content, or how peculiar they may "look” to the critic, are 
actually measuring the function in question. This is the method 
that has been followed in the case of the Kelley-Trabue character 
measures. 


[ 149 ] 



TESTS AND MEASUREMENTS 


We may illustrate by one of the eight traits which it was at- 
tempted to measure. It is conceivable that exercises, perhaps 
somewhat like the sample below, might be built up to measure co- 
operattveness of pupils. 

Item E,/of a fourth grade group: 

Four children were building with lumber and blocks in the yard. Each made 
a remark. The first said, 

“Let me have my share of the blocks and I will build in the comet." 

The second said, 

"Let us all build together and make a house." 

The third said, 

"1 want to build a great big tower.” 

The fourth said, 

"When you get through building I will be a hurricane and knock down what 
you have built." 

^ Which of the four remarks is most like a remark that you yourself might have 
said? Put a check mark in front of it. 

Our adult judgment might assert that to check the second is 
evidence of cooperativeness. Perhaps this is correct, but the con- 
struction of appropriate exercises is extremely difficult, and the 
items axe likely to reveal the intent of the test and thus vitiate it. 
There are so many pitfalls that it is hazardous to trust one’s judg- 
ment in this matter. On the other hand, if a school teacher knows 
the four children in question, he can with considerable certainty 
rank them for their general spirit of cooperativeness. Judgment is 
here upon fairly trustworthy ground and represents a sort of aver- 
age opinion based upon various contacts with the pupils. Judg- 
ment of an item, however, is very likely to be a judgment in vacuo, 
the item not having come within the direct experience of the person 
judging. This, for example, would be the case if a teacher had 
never observed the conduct of enough children who had said, 
“I want to build a big tower,” to know whether it was a coopera- 
tive or a non-cooperative attitude that, Was being manifested. 

In fields not characterized by a substantial and recorded tra- 
ditional content the a priori construction of test items offers such 
difficulties and uncertainties that it is doubtful whether it should 
be attempted} or, if attempted, it should be non-determinative of 
finsd test construction outcomes unless verified by an independent 

{150] 



VAUDITY CRrrERION 


Study of validity. It was not attempted in the process of securing 
measures somewhat similar to cooperativeness, such as courtesy, 
fair play, honesty, but a procedure considered to be much sounder 
was followed. Teachers were asked to rank pupils for courtesy. 
Then a free association test (wherein a child is called upon to 
respond with the first woi^ coming to mind after seeing and hearing 
a stimulus word) was given. The responses were then classified 
according to the judged traits of the pupils making the responses. 
If the stimulus word is “school” and a number respond with 
"mates,” and if the average courtesy ranking of such is distinguish- 
ably (t. e.y sufficiently so that chance is a probable explanation) 
above the average of pupils in general, there is indication that to 
respond with the word “mates” is indicative of courtesy. The weak- 
nesses of this method of determining what are courteous responses 
are the size of the chance factors involved, the laboriousness of 
determining the significance of each of the very large number of 
responses made, and, finally, the laboriousness and cost of scoring 
the responses of a newly tested child. 

This is not the place to discuss these difficulties in detail or to 
suggest how each might be met. It here suffices to point out that 
a very different criterion is being employed, namely, judgments of 
competent acquaintances of individuals, not judgments of test 
items. With this procedure there can be no question (granting 
computational accuracy) that the trait being measured by the 
responses as scored is the trait indicated by the test label — cour- 
tesy. Certainly courtesy means that trait in individuals that is so 
characterized by competent acquaintances, and nothing more and 
nothing less. Accordingly in the measurement of character traits 
by means of the free association test here undertaken, there was no 
call for, nor dependence, upon, judgments of social scientists op- 
erating upon test items, but there is utter dependence upon the 
appraisal of traits of individuals made by their acwaintances, i. e., 
teachers and fellow pupils. w 

In summary: Tests of the social concepts s<m. are considered 
valid because (1) current usage provided the terms, (2) social 
scientists certified to the substantial accuracy of the distinctions, 
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or meanings, found in each test item, and (3) average progress 
from ip-ade to grade was substantial. 

Tests of historical (Marion Clark), and geographical (R. D. 
Calkins and Edith Parker) abilities, and of attitudes (L. L. Thurs- 
tone), are considered valid because (1) of the competence of the 
social scientists defining the functions tested, and {2) of trust in 
the ability of test authors to devise exercises measuring these 
functions. 

Tests of character traits (T. L. Kelley and M. R. Trabue)^ are 
considered valid because (1) those possessing a trait are, except 
for chance errors of judgment, those thought by acquaintances to 
possess it, and (2) the scoring scheme built up is such that the test 
reveals, except for chance errors, just the trait differences asserted 
by the acquaintances. 

This covers the types of criteria used in the construction of the 
various sorts of social studies tests. A more detailed illustration 
of their use is to be found in connection with the separate tests. 

‘See Chap. V. 
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TESTING OF UNDERSTANDING 

The list of objectives drawn up by the planning committee de- 
scribed the objective of understanding as follows: 

‘‘Understanding of important institutions by means of which 
society functions. Principles and ideals 'are included. 

This is the most generally accepted function of the social 
studies, and the most precise. Understanding and informa- 
tion are not synonymous; and tests should seek for under- 
standing. To test this objective effectively, it will be neces- 
sary to break it up into parts. 

Political institutions — local, national, international; principles 

and ideals. 

Economic institutions, local and general; principles and 

ideals. 

Social institutions, domestic, ethical and religious; principles 

and ideals. 

Educational institutions — principles and ideals. 

Esthetic institutions — principles and ideals. 

Recreational institutions — principles and ideals. 

These institutions, principles, and ideals, are to be drawn 
from the standard text-books and syllabi at research centers. 
The material thus collected is to be used as the basis for the 
construction of tests. 

In 1929, when the work on test construction began, it seemed 
highly desirable to break up this objective as minutely as possible, 
into single words or at the most very brief phrases. The limita- 
tions which this procedure involved are discussed elsewhere.^ To 
discover single words which might be used to test understanding 
rather than memoriter information suggested some distinction. 
The pioneer work of Van Wagenen, the work of Woody and Steph- 

‘See Chapter I., pp. 5-18. 
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enson» of Mf|| Pkessey and of Harold Rugg’s students was studied 
for possible dues. I'inafly it was decided to distinguish terms desig- 
nating direct relationship from those which designated persons, 
objects, events, and circumstances related. Most proper nouns and 
words were thus eliminated and attention was concentrated chiefly 
upon common nouns and terms. A list of such terms was compiled 
from standard textbooks used in the social studies through the 
junior college years. ^ 

The authors of the tests of undfrjtanding constructed on this 
basis for the Commission have described their work in the follow- 
ing essays. Wesley and Gold worked at the secondary school level, 
Wesley continuing the work on the junior college level. Kelty con- 
structed tests for the intermediate grades which could also be used 
at the junior high school level. These three worked immediately 
under the direction of the adviser on tests. All are teachers of the 
subject-matter. As a check upon their work, Mrs. Pressey, a psy- 
chologist long interested in this problem as one of educational meas- 
urement, constructed her tests independently. The comparison of 
results was to prove interesting and instructive. ‘ 

*See Appendix 1 for this list. 

*See Chapter I, p. 37 fF. 
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A STUDY IN THE LEARNING OF THE FUl^AMENTAL 
SPECIAL VOCABULARY OF HISTORY FROM 
THE FOURTH THROUGH THE 
TWELFTH GRADES 

Luella Cole Pressey 

1 

Ohio State University 

1. Previous Research Used as a Basis for the 
Present Study 

For about ten years the writer has been in charge of remedial 
work done with college students who were on probation because 
of academic failures. From the beginning of the work it became 
obvious that many of these probation students had fallen into 
scholastic disfavor because they could not read their textbooks 
intelligently. This problem of reading was especially acute in the 
social sciences. In analyzing the difficulties involved, the writer 
inevitably and quickly came upon the paucity of vocabulary dis- 
played by these students as one of the major causes of inadequate 
comprehension of what was read. It seemed evident then that a 
step in the remedial program would be the building up of a more 
adequate vocabulary. However, even a casual glance at textbooks 
in the social sciences revealed literally hundreds of special words, 
quite evidently varying greatly in frequency of appearance, im- 
portance, and difficulty. The task of teaching the meanings of all 
such words to inefficient readers of social science texts was ob- 
viously both impossible and unnecessary. The writer therefore set 
about finding out which were the essential technical words wdth 
the idea of teaching these first. This research turned out to be a 
real task, requiring some eight years for even the solution arrived 
at to date. 

In determining which were the really essential words the writer 
anal}rzed first twenty-three textbooks in history and civics used in 
grades from the fourth through college, and listed what words 
occurred, without noting the frequency of each word. Later she 
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analyzed aix hiatory texts widely used in the hi^ schools, this 
time keeping a count of how many times each special word oc- 
curred. Another step was to make use of published results of other 
similar analyses in history and civics.^ Finally, frequency counts 
were made of the special words occurring on the front pages of 
newspapers, in the editorials, and in many articles in magazines 
dealing with current events. 

From a combination of word lists culled from these various 
sources a master list of 1,444 diferent words was constructed, the 
frequency of each word appearing after it on the list. The matter 
of mere frequency of appearance was one criterion used in select- 
ing the essential words. 

In addition, sixty-four teachers of secondary school history and 
five professors of college history rated each of the 1,444 words. 
They marked each word in one of three ways: “essential,” if they 
thought they could not teach without it; “accessory,” if they re- 
garded the word important but not essential; or “unimportant,” 
if they thought they could get along without it. A combination of 
these ratings served as the second criterion. 

The entire list of words was next rated by seven especially 
trained individuals* who expressed their feeling as to the “socio- 
logical value” of each word — that is, its use outside the history 
classroom. These ratings constituted the third criterion. As a final 
step, the writer went over the total results from all three types 
of investigation — frequency, importance, and sociological value — 
and selected those words that were frequently used, considered 
important by teachers, and of high value outside the classroom. 
Any words not meeting all three criteria were eliminated from this 
list of absolute essentials. The selected list contained 415 words 

*Bairr, A. S., and Gifford, C. W., "TTie Vocabulary of American Histom” 
7^ Journal of Educational Rtsearck, XX: 103-121, 1929. Also Stephenson, A. W., 
'The Special Vocabulary of Civics,” The Journal of Educational ^search, XVIII: 
297-304, 1928. 

*A coHeM ptofessor whose specialty is professionalized subject-matter in his- 
tory, one cmeae professor of history, an experienced and intelligent social worker, 
a mature high smool prindpal who had taught histop^ for several years, a public 
school specialiat m educational research, a specialist in test construction, and the 
wnter. > 
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(of which, however, only 346 were actually tested, the remaining 
69 being omittted for one reason or another, used as examples, or 
appearing in unsatisfactory test items). The list may be found in 
Appendix A at the end of the paper. 

The next step was to construct a test by means of which com- 
prehension of these minimal terms might be measured and specific 
shortcomings discovered*, so that remedial work might be done. 
An objective form was used because of its high reliability* and the 
great convenience in its administration. Special effort was made 
in the construction of the test questions, to avoid mere definitions 
and to present each special word in a contextual setting that would 
not be too unlike the setting in which it might appear in a text. 
A few examples are presented below: 

11. What is sometimes given to people to prove their ownership 
of the land they settle upon? (a) petition, (b) charter, (c) 
doctrine, (d) amendment. 

S3. What happens when the value of money is inflated? (a) it 
will buy more, (b) it is not possible to invest it, (c) it can 
be used to pay international debts, (d) it will buy less 

68. When is an injunction most often used? (a) during a strike, 
(b) during a battle, (c) during an expedition, (d) during a 
voyage 

76. When a law is repealed what is always the result? (a) it is 
enforced more rigidly, (b) it is amended, (c) it is no longer 
in force, (d) it is a source of dissatisfaction 

The work had already been completed up to this point before 
the grant was made by the American Historical Association for 
the research reported in the main part of this paper. With the be- 
ginning of the research under these auspices* the first step was to 

‘Reliability means simply that the scores on one form of a test correlate wdl 
with scores on another form; that is, the same pupil tends to make the same score 
on twoTorfmore forms covering the same material. The average coefficient of reli- 
ability for'single grades containing 160 to 190 pupils of the first 80 items of the test 
constructed as compared with the second 80 is .89; that between the third 80 items 
and the last 80 is .%. These coefficients of reliability are soiiigh that there can Iw 
no question as to the reliability of the materials. The matter of validity — that is 
the question as to what the test actually measures — will be returned to later. 

*More directly, under the supervision of the Commission on the Investigarion 
of the Social Studies in the Schools. 
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have representatives from the various social sciences go over the 
questions to be used, with the idea of making them as acceptable 
as possible to teachers in these subjects. The writer’s original for> 
mulations of 325^ questions were, therefore, gone over and critized 
by eight people from the social sciences. The final formulation of 
each question is more or less a compromise among the various 
points of view. The writer is quite willing to admit that the word- 
ing is not always satisfactory; in fact, three items had to be thrown 
out of the results because they were too poorly presented even to 
score. The remaining questions, although not perfect, seemed to 
be well enough phrased and to have good enough “confusion” an- 
swers (with a single, unmistakably right answer) to yield informa- 
tion of interest. 

For convenience in handling, the 325 test questions were pre- 
sented in four four-page folders, with 85 items in the first folder 
and 80 in each of the other three. These folders will hereafter be 
referred to as Forms A, B, C, and D. The selection of questions 
for each form was supposedly entirely random (as was also the 
order of questions within each form), but as a matter of fact the 
student who made up the forms did not follow instructions exactly, 
with the result that an undue proportion of the political terms got 
into one form, too many of the military into another, and so on. 
However, the forms are of no separate significance and were used 
merely for ease in testing. 

II. Purpose of the Investigation 

The chief purpose was to trace the learning of each of these 
essential words from the low fourth grade through the twelfth. 
It was also desirable to observe the general trend of the learning 
of the total vocabulary. When the mastery of each item of special 
vocabulary had been shown, it was the writer’s intention to com- 
pare vocabulary achievement at each level in school with analyses 
of textbooks used, to see what relation there might be between 
Vrards actually known and words assumed by the texts as known. 

ifiecauw a lew questions involved die use of more than one of the special 
word** there weni fwer questions than there were terms. 
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A further comparison of words appearing frequently in newspapers 
and current magazines with mastery at the end of the twelfth 
grade seemed desirable as a means of estimatidg how complete the 
training in school was — provided a pupil finished high school — as 
a preparation for the reading of current history. There were, then, 
two major objectives: (1) to determine the progress of mastery 
through the grades studied, and (2) to compare this mastery with 
(a) the vocabulary used in the textbooks read at various levels 
and (b) the vocabulary needed for intelligent reading of newspapers 
and magazines. 

For the purposes just enumerated the materials are subject to 
certain obvious criticisms which the writer prefers to consider be- 
fore taking up the results obtained by the testing. In the first 
place, (1) the results show only recognition of the meaning of the 
words and not spontaneous definition. However, “merely recog- 
nition” is exactly what happens when one reads; perhaps this ele- 
ment in the situation is an advantage rather than a handicap. 

Secondly, (2) the test obviously measures comprehension of each 
term in the single context presented and does not tell the extent 
to which these same terms (a) would be recognized in equally good 
but differently worded test items, (b) would be understood in con- 
text when an assignment was being prepared, or (c) would be 
defined correctly in spontaneous writing. Studies of the agreement 
of duplicate test items show, however, high correspondence be- 
tween the duplicates; thus it may be supposed that the per- 
centage of children correctly answering these questions will not 
vary greatly from the percentage that would respond correctly to 
another test item of similar objective type. 

The question as to the extent to which recognition of word 
meanings in a test corresponds with similar recognition while read- 
ing a textbook is obviously harder to investigate. The writer has 
here only two studies of her own to offer. In one, the relationship be- 
tween correct scores on 40 vocabulary items correlated with com- 
prehension of passages containing these vocabulary elements with 
a coefficient of .79. In the other, the correlation between marks in 
history (depending presumably to a great extent upon reading) 
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and scores oa a history vocabulary test (for 387 cases) turned out 
to be .67. 

As regards the relationship between spontaneous definition of 
terms and recognition of these same terms in objective form the 
writer has one further study. In history classes 387 students were 
asked to define 25 words selected from the tests. These responses 
were graded. About a month later these same students were given 
the objective tests. These responses were also scored. The results 
were then summarized in terms of the percentage of agreement 
between written definitions and recognition. It was found that 
there was perfect agreement in 70 per cent of the instances; that 
is, 70 per cent of the time those who answered a given test item 
correctly also defined the words involved correctly, and those who 
failed an item were unable to give satisfactory definitions for those 
particular words.* On the whole the writer feels that what evidence 
there is points to the conclusion that recognition of meanings in 
one setting is more likely than not to be directly associated with 
recognition in another setting. 

(3) Qearly, a test item can test only one meaning of a term. 
The writer is quite willing to admit that the particular meaning 
of a test item that is offered is not always the one most frequently 
used, not always the best one, and sometimes is a colloquial, rather 
than a rigidly technical, meaning. Some of the meanings are at a 
very elementary level; others require far more mature understand- 
ing. If anyone else had^been mainly responsible for the items, the 
same situation would, in all probability, have appeared. However, 
it must be remembered that “comprehension” inevitably means 
“comprehension as called for by this or that particular question.” 
This limitation is not due entirely to the objective form; it enters 
into any question asked, whether the answers are supplied or are 
to be written out spontaneously. 

(4) As mentioned earlier, a few items proved to be entirely un- 
satisfactory; that is, they had no clearly correct answer or else 
^td more than one equally good answer. Results on these items 
hkTe been omitted. ^ 

' *Tlnt material is to be published later by Dr. Elinor Barnes and the writer. 
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In spite of the shortcomings, readily acknowledged, the writer 
feels that the results presented are significant and of practical 
value to teachers. 


111. Methods of Procedure in the Investigation 


The data presented belpw are based on results from 16,682 tests 
taken by over 11,000 pupils in the fourth, sixth, eighth, tenth, and 
twelfth grades of thirty-one different places. There was an aver- 
age of 4,171 cases per form and 834 cases per grade. The number 
of cases per form and per grade, as well as the geographical dis- 
tribution of the places cooperating^ in the project is shown by 
Table I. 

At this point recognition should be made of one technique used, 
without which the testing of such a large number of cases with 
the money available would have been impossible. In the first place 
only 4,000 blanks were printed.* None of these were marked at 
all (with the exception of a few used to get comparative data as 
to expense) and so they could be used over atid over again. What 


’^The writer is indebted to the following persons for their assistaiue in gathering 
the data: Dr. Brian E. Tomlinson of Emporia State Teachers College and the 
superintendents of Americus and Emporia; Dr. H. H. Bixler of Atlanta, Georda; 
Dr. Virgil E. Dickson of Berkeley, California; Mr. W. C, Kennard of Harvard Uni- 
versity and the school officials of the Cambridge high schools, and ofWenham, Bever- 
ley, and South Hamilton, Massachusetts; Principal Elizabeth Fassi^ of Columbus, 
Ohio; Dr. Charles £. Greene of Denver, Colorado; Dr. L. V. Cavins of Charles- 
town, West Virginia; Dr. Arthur Repp of the State Teachers College at FlagstaflF, 
Arizona; Superintendent S. M. Brownell of Grosse Point, Michigan; Mrs. maiy 
LakoflT, Supervisor of Social Studies in Hamtramck, Michigan; Superintendent 1. 
M. Allen of Hi^land Park, Michigan; Dr, W. W. fCemmerer of Houston, Texas; 
Dr. Anna G. hG'crs of Kansas City, Missouri; Dr. 1. N. Madsen of Lewiston State 
Normal School, Lewiston, Idaho; Dr. Ernest Branson of Long Beach, California; 
Dr. Dean A. Worcester of the University of Nebraska and Superintendent Lefler of 
the Lincoln, Nebraska, schools; Dr. W. W. Coxe of the State Department of Edu- 
cation in New York; Superintendent E. E. Jones of Mechanicsville, New York; 
Dr. W. W. Theisen of Milwaukee, Wisconsin; Dr. J. J. Forester of Montclair, New 

Van, r' 


Jersey; Superintendent C. G. Hethcrington of Penn Yan, New York: Dr. Philip A. 
Boyer of Philadelphia, Pennsylvania; Dr. C. £. Myers of the State Board of Edu- 
cation in Virginia; Supervisor R. V. Bagby of Roanoke College, Virdnia; Dr. James 
F. Bursch of Sacramento, California; Dr. Virgil Smith of Seattle, Washington; Dr. 
C. W. Odell of the University of Illinois and the Superintendent of the schools of 
Urbana, Illinois; and to Dr. E. S. Henry of the New York University, New York 
City. 

*The printing was done through the courtesy of Ohio State UniversiQr and 
without expense to the Investigarion. 
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TABLE I: Number of Caeei for Each Grade on Each Form of the PrelmuiMrj' 
Teeti, and the Geographical Distribution of the Cooperating Cities and Towns.’ 

Form A Form B 



Beverly, Mast 163 143 191 164 

Chatlettown. W. Va 97 116 156 136 ... 94 115 123 134 

Col lunbut, Ohio 120 126 

Flagstaff, Aria. 11 10 

Groste Pointe, Mich 271 . . . 149 

Hamtramck, Mich 196 824 340 176 

Highland Park, Mich ISO 244 197 189 

Lewiston, Idaho 93 124 90 63 ... 94 127 90 60 

Lincoln, Neb 177 157 299 

Mechanicville, N. Y 135 ISO 108 48 

Milwaukee, Wis 326 ... 70 

Montclair, N. J 22 37 ... 23 22 

New York City 46 39 

PennYan,N.Y 100 72 34 60 

Sacramento, Cal 713 720 

So. Hamilton, Mass. . . 32 25 22 27 ... 30 25 24 26 . . . 

Wenham, Mast 14 13 10 14 12 12 


212 457 2416 926 1018 209 483 1397 926 976 


Form C 


Form D 



4 

6 

8 

10 

12 

4 

6 

8 

10 

12 

Americas, Kans 

... 

11 

7 

23 

IS 

... 

11 

7 

22 

IS 

Atlanta, Ga 

, , , 

a a a 

266 

a a a 

a a a 

a a a 

a a a 

270 

a a a 

a a ■ 

Berkeley, Cal 

e « • 

a a a 

a a a 

a a 

191 

a a a 

a a 

■ a 

a a 

182 

Beverly, Mass 

e ■ . 

a a a 

a a 

141 

184 

a a a 


a a 

138 

165 

Columbus, Ohio 

128 

a a a 

a a a 

a a a 

a a a 

117 

a a 


a a a 

a a a 

Emporia, Kans 

• • • 

32 

64 

45 

26 

a a a 

32 

65 

22 

14 

Houston, Tex 

• • • 

a a a 

a a a 

a a a 

98 



a a • 

a a 

99 

Kansas City, Mo.. . , 

• • • 


a a a 

205 

247 


a a 

a a 


a a a 

Long Beach, Cal 

a > t 

254 

265 

228 

248 


250 

230 

230 

247 

New York City 

43 

a a a 

a a a 

a a a 

a a a 

41 

a a a 



a a a 

Philaddphia, Pa 

a ( a 

e a a 

a a a 

152 

91 

a a a 

a a a 

a a 

146 

88 

Roanoke Va 

Seattle, Wash 


a a a 

473 

a a a 

a a a 

• a a 

a a a 

464 


a a a 


125 

146 

a . a 

a a a 


122 

143 


e a a 

So. Hamilton, Mass. . . 

29 

26 

25 

23 

eat 

28 

26 

24 

25 

t a a 

Urbana, IB 

a * * 

a a r 

276 

a a a 

a a a 

a . a 

a a 

274 

a a a 

e a a 

Wenham, Mass. 

14 

12 

12 

... 


15 

14 

11 



214 460 1534 817 1100 201 455 1488 583 810 

iln tdi£tini,w|SK rswiTed, too Iste to be induiied in the tablet, 468 catei, euhth grade, 
PreHnrinaty F«caa A and 470 caaet, eighth grade, Prdiinmary Form B from Denver, Colorado, and 
37 catae, teath gradi, PreUttinary Form D, fram Cambridge, Mattachutena. 


[1621 



METHODS OF PROCEDURE IN THE INVESTIGATION 

each child actually marked was a 3 by 5 indi card* on which there 
appeared spaces for writing down the letter that stood before the 
answer he selected as right. On this card he wrote also his name, 
age, grade, and other such information. A copy of the first two 
lines of this card appears below: 



PRESSED POCKET TESTER ® AMEXICAW education niEW, Caluttn, oil* 


The value of this technique is twofold. The cards are much less 
expensive than a separate blank for each child. And the labor of 
scoring and tabulating is greatly reduced. Thus the entire 16,682 
cards® were scored and tabulated by item for a total cost of 
3255.65, or barely IK cents a card (.0153ji). In contrast it may 
be said that the same scorers similarly handled results from the 

*Printed by the American Education Press, Columbus, Ohio. 

“There were 3,61S other cards used in odd-numbered wades; these were 
scored but not tabulated by item. They were not included in the above number. 
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Tasli II: Diitribution of Total Scorea or Prelitiiinaty Form A of die Vocabalaiy 

Tetr 


Score 



GRADES 



4 

6 

8 

10 

12 

80 






78-79 




5 

37 

76-77 



5 

16 

91 

74-75 



5 

18 

123 

72-73 

.... 


IS 

42 

156 

70-71 

.... 


27 

56 

123 

68-69 




35 

79 

89 

66-67 



66 

64 

98 

64-65 

.... 

i 

93 

57 

75 

62-63 

.... 


93 

82 

SO 

60-61 


2 

128 

71 

51 

58-59 

1 

1 

132 

69 

30 

56-57 


S 

141 

55 

25 

54-55 

. . . . 

3 

130 

46 

15 

52-53 


5 

137 

46 

15 

50-51 


8 

115 

50 

5 

48-49 

i 

7 

106 

52 

8 

46-47 


16 

119 

27 

7 

44-45 

2 

10 

131 

21 

6 

42-43 


18 

120 

12 

2 

40-41 

1 

20 

111 

11 


38-39 

. . . . 

19 

116 

10 

2 

36-37 

1 

22 

105 

11 

2 

34-35 


18 

94 

12 


32-33 

2 

30 

77 

3 

3 

30-31 

s 

44 

96 

1 

1 

28-29 

4 

27 

72 

2 

1 

26-27 

S 

39 

47 



24-25 

8 

41 

47 

1 

1 

22-23 

13 

31 

21 



20-21 

10 

28 

13 

1 

2 

18-19 

22 

12 

9 

1 


16-17 

32 

14 

5 

1 


14-15 

22 

16 




12-13 

18 

9 

1 


, , 

10-11 

13 

2 

2 



8-9 

16 

3 




6-7 

11 

4 




4- S 

11 

1 




2- 3 

6 

1 




0- 1 

8 





Total 

... 212 

457 

2,416 

926 

1,018 

Me£an 

... 16.0 

30.0 

48.4 

60.8 

70 4 
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two school systems in ^ich the blanks were marked at a rate of 
cents per test; this rate would have required $750.69 for hand- 
ling the same total number of cases (plus the greater expense in 
printing, since each child would have n^ded a blank). 

IV. Results or the Investigation 

The record cards were first scored for the total number of items 
correct. The results from a single form* are presented on p. 164, to 
show the nature of these results. The wide variation of scores 
should be especially noted as indicative of the individual differ- 
ences in attainment shown among children classified by the school 
as being of the same educational level. Thus, in the fourth grade 
there is one child who tests near the median for the tenth grade, 
another who tests at the median for the eighth, and eleven who 
test above the median for the sixth. In the tenth and twelfth 
grades together there is one child who tests at the median for the 
fourth grade, an additional nine who test at or below the median 
for the sixth, and one hundred and thirty-one more who test below 
the median for the eighth. In Grade VIII the range is from almost 
complete failure to almost complete success. The medians also are 
interesting. The largest gain is between the sixth and eighth 
grades, with smaller gains between the fourth and sixth, the 
eighth and tenth, and the tenth and twelfth. This characteristic 
of the results will be returned to lat^r. 

While the separate forms of the tests have no significance be- 
cause of the largely chance arrangement of the items, the sum of 
the medians on the four forms will give a rough indication of the 
degree of mastery of the 310‘ items in the different grades. Sbce 
with a very few exceptions, the same children did not take all 
tests, no total score on the entire examination is possible, but an 
approximation may be arrived at by adding together the medians 
for the four forms; this total should not be far from the truth, since 

^Distribution for all forms will be found in Appendix C at the end of the paper. 
*Of the 325 original items, 10 were used as examples, 3 were unsatisfactoty, 
and 2 had to be dropped because of an enor in the score sheet. 
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the results for all forms are based on equally unselected groups. 
The results of this addition appear below: 

TABLE III: Summitioa of the Mediam for Each Grade oa Each Preliininaiy 

Form of the Tettf‘ 


Grade 4 Grade 6 Grade 8 Grade 10 Grade 12 


Form A 16.0 30.0 48.4 60.8 70.4 

FormB 26.1 39.2 55.5 63.4 70.3 

FormC 18,3 32.0 45.9 57.2 68.2 

FormD 22.0 37.4 54.0 61.8 71.4 


Total 82.4 138.6 203.8 243.2 280.3 

Difference 56.2 65. 2 39.4 37.1 


rit will bt noticwl thit the fomii ere not ettonther cquet in difficulty. However, the Urtett 
vtriltkm il due to the feet that Form C had only 74 itemi that were acocnl, and Forme A and D 
79 hew, while Form B had 80. Any further variation ii due to the fact already mentioned that 
tha maietant in malimc up the forma did not uac an altogether random aelection. Since the total 
teen on any one form ia of no aa^ial aigoificance the matter ia of little importance, but it doei 
otplaiB the Ttriability in the meoiana. 


In considering the increase from one of these levels to the next 
it is clear that the main improvement takes place between Grades 
IV and VIII, the additional amounts being only a little more than 
the rise from Grade VI to Grade VIII by itself. This failure of 
the high school grades to show more marked gain is not due to a 
lack of room for improvement; the Grade XII median is still 
thirty pmnts below the total number of items in the examination. 
The lack of growth in vocabulary in the upper grades is especially 
significant because most pupils have one year of history in high 
school and many, of them have two (plus more or less work in other 
social sciences). The situation is probably due in part to a differ- 
ence in attitude toward the problem of vocabulary; the teachers 
in the elementary grades assume few words as known and conse- 
quently make specific efforts to teach vocabulary, whereas those in 
high school tend to ignore the problem. It is also natural that, the 
easiest and commonest words having been learned, mastery of the 
harder and less frequent words should be slow. 

After the record cards had been scored and the total scores 
handled as shown above, the correct answers were tabulated by 
item for each grade in each school. The number of children pass- 
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tng each item in each grade on each form for all schools was then 
found and these numbers turned into percentages of children in 
each grade knowing each item. At this point the items from the 
various forms were rearranged and the responses classified in 
terms of the words being tested. The list of 346 words, knowledge 
of which was required in the 310 items,* with the per cent recog- 
nizing each word in each grade began as follows: 

TABLE IV: The Percentage of Children in Each Grade Recognizing the Meaning 

of Each Word 


Words Grade 4 Grade 6 Grade 8 Grade 10 Grade 12 


ambassador (used as example and not scored) 

authorities 21 40 81 87 94 

consul 19 38 61 83 87 

governor 61 61 84 88 97 

king (item omitted because of error in scoring key) 

minister 11 26 32 32 51 

etc. 


It should be noted that this list of words corresponds exactly 
to the list previously referred to as the outcome of the research 
into what constitutes the essential terms, which list is presented 
in full at the end of this paper. Each word in that list is accounted 
for in some way in this list of results, which appears in its entirety 
later on, p, 208. 

The next step was to consider once again, from a study of the 
gain in percentages shown by the 346 words tested, the general 
growth in vocabulary. The percentages shown by all words in 
each grade were therefore tabulated, with results as shown in 
Table V. 

This table shows the same general trend as the earlier ones — 
that the most significant rises come in the lower grades. It shows 
also the spread of vocabulary within a given grade. For instance, 
there are three words known by 80 to 84 per cent of the pupils even 
as low as the fourth grade, and another twenty-one known by over 
50 per cent. On the other hand, ten words were recognized by less 

*It should be pointed out that when two terms are tested by the same item 
the resulting percentages reflect presumably the mastery of the harder word. 
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TABLE V : Percenuge of Gtildrea in Excb Gnde Annrering Each Itom Correctly ^ 
alao the Spread of Mutery of die 346 Wordi in Each Grade 


Per cent 



GRADES 



4 

6 

8 

10 

12 

100 





1 

95-99 


2 

22 

43 

110 

90-9i 


3 

33 

63 

61 

85-89 


8 

29 

51 

45 

80-84 

3 

17 

36 

24 

33 

75-79 

2 

9 

26 

27 

27 

70-74 

1 

22 

24 

24 

IS 

65-69 

1 

11 

IS 

21 

12 

6044 

5 

24 

22 

13 

11 

55-59 

3 

12 

23 

24 

8 

50-54 

9 

23 

21 

14 

7 

45-49 

11 

23 

18 

7 

4 

40-44 

19 

29 

22 

8 

2 

35-39 

23 

26 

11 

9 

S 

30-34 

36 

31 

8 

12 


25-29 

48 

36 

15 

1 

1 

20-24 

41 

30 

9 

3 

4 

15-19 

56 

21 

9 

1 

, , 

10-14 

51 

15 

3 

1 


5-9 

27 

4 




0- 4 

10 

• « 

. . 

, . 

• • 



— i-i— . 

'■ 1 




Total Number of 

Item! 

346 

346 

346 

346 

346 

Percentage of Chil- 
dren Knowing 

Median Item .... 

23.5 

41.7 

69.0 

81.6 

90.0 


than 5 per cent of the pupils. The sixth grade shows words varying 
from two known by 95 to 99 per cent of the children to four known 
by less than lO per cent. It is interesting to note that even in the 
tenth and twelfth grades there are still words known by less than 
half of the pupils— 42 such words in Grade X and 16 in Grade XII. 
^though the percentage of children knowing the median word for 
ead) grade rises definitely, there is a marked lagging behind of cer- 
tain words. If these terms could be isolated and taught, the 
mastery could be considerably improved with relatively little 
effort. At tlie other extreme, it is encouraging to see that by the 
mid of high school 172 wbrds are known to 90 per cent or more of 
the diildren. 
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If one were to consider as really mastered those words recogniEed 
by 90 per cent or more of a class, then there would be no such 
terms in Grade IV, five in Grade VI, fifty-five in Grade VIII, one 
hundred six in Grade X, and one hundred seventy-two in Grade 
XII. With this standard* almost exactly half the total 346 words 
were "really known” at the end of high school.* 

As a final step in presenting the results the writer constructed 
graphs for each word, using the percentage of children passing each 
item in each grade as a basis. The entire number of graphs will not 
be presented because they would occupy too much space, because 
they are of a limited number of types easily demonstrated by a selec- 
tion, and because anyone interested in studying them can easily 
construct them for himself from the percentage table given at the 
end of the paper — the same table the author used. However, cer- 
tain typical graphs are given below, so that the general nature of 
the results may be appreciated. It should be noted that these 
graphs show learning curves for the term being tested. If the 
reader wishes to know the exact context of the terms thus illus- 
trated, he is referred to the table of percentages in Appendix D, 
where he will find the designated item in which each word is to be 
found; he can thus look up the actual questions on the test blanks, 
which also appear at the end of the paper, and have the question 
before him. 



^Anyone wishing to set a standard at a point other than 90 per cent can do 
so by use of the detailed table giving percenuges in every grade for every word. 
*A list of these words appears later, p. 175 S. 
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The firft group of graphs illustrates a common type of learning 
curve — the straight line growth from low mastery in Grade IV to 
high mastery in Grade XII. There were seventy such graphs. 

These three terms show gains from the bottom to the top of 65, 
71, and 74 respectively. 

The next four sets of graphs show variations on the above funda* 
mental type. The graphs for the learning of the words “domestic,” 


Cfdss— — — Qrades ■ '■« —— Grades 



“powers,” and “lobbying,” show a slow start, but a straight line 
increase once the learning has begun. These concave curves (in 
number 31) are typical of the more difficult words. 

A variation in the opposite direction is shown by the curves 
representing mastery of the words “agriculture,” “naturalization,” 
and “strike.” For these words the growth comes in the early 
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grades with almost perfect recognition thereafter — thus giving a 
convex curve of which there were 32 instances. 

A third variation is illustrated by the three curves below. 


■Grades i ..i ■ 1 1. Gradea — Gradei 



The mastery of these three words is already so high in Grade IV 
that little growth is possible, but what growth there is takes place 
regularly. They represent really the top of a learning curve whose 
beginnings are somewhere below Grade IV. Words showing a 
mastery of 40 per cent or more in the fourth grade and then 
increasing regularly are included in this group; there were 45 
such words. 

The words "sedition,” "abdicate,” and "militia” illustrate one 
more variation. In these cases the growth is again perfectly regu- 
lar, but so slow that the final achievement in no instance exceeded 
54 per cent. There were 36 curves of this type. 

. .1 ...Grades Grade* ——Grade*—— 
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The curves thus far have shown regular growth. There remain 
four more types of curves in which growth is, for some reason, 
more or less irregular. Those for the words “premier” and “execu- 
tive” contain marked plateaux. What learning took place oc- 



curred early, the later grades merely holding their own. In the 
case of “contraband” there is again a plateau, but this time from 
the sixth through the tenth grade, with Grade XII marking a final 
slight rise. There were 19 curves showing plateaux at some point 
in their rise (and below 90 per cent, after which a plateau is to be 
expected). 

The three curves next presented are interesting because of the 
failure of the upper grades to make the expected gain. The Grade 
IV results start these curves off with a relatively high achievement 

— Qr»dM«— — ——Grades— ■ •. Gradea ■ ' ■ 
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— ^from 31 per cent to 47 per cent in this earliest grade knowing 
these words. In spite of this good start, the final standing in no 
case exceeds 73 per cent. The total gains are only 26, 30, and 27 
per cents. These words need study to find out why they do not 
show more rise. There w^re only 6 words that acted in this way. 

The next group of curves (which contains 80 instances) illus- 
trates a rate of learning commonly found in this study wherein the 
rise in total score from the sixth to the eighth grade always ex- 
ceeds that between any other two grades. There are more curves 
of this shape than of any other. In each of the three instances 

— — Qrsdet-— Grades — Qredee — 



there is little learning before Grade VI and not a great deal after 
Grade VIII, but an increase of 50 per cent or more between these 
two grades. If the reasons for the splendid achievement at this 
level could be isolated, and the techniques applied to learning at 
the higher levels the problem of vocabulary might be solved. 

Finally, there are few words showing such results as are illus- 
trated below. These can hardly be called learning curves in the 
absence of both learning and curve. The mastery in all grades is 
low, nowhere exceeding 35 per cent. As may be seen from the 
table already presented (Table V) these results are not numerous 
— in fact, there were only 14 curves of this type. 

In addition to these well-defined types of curves, there are a 
few words that show peculiar results in that a later grade scores 
lower than previous grades. There seems no plausible explanation 
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■OrsdM* 


■Gr»dM< 



for these findings. There are, however, only 13 such words, and 
in no case is there more than one grade that shows such variation. 
All other words show every grade at least holding its own with the 
grade below and usually showing improvement.* 

Consideration of any or all of the results presented above leaves 
one with the conviction that not enough special vocabulary is 
acquired by the average student at any level to serve as a sure 
foundation for the reading of the average textbooks used in these 
grades. Again it should be pointed out that these words are, for 
the most part, of fundamental character. Words less frequently 
used and less important would presumably show even less mastery. 
The achievement shown is, then, the "cream” — and one wonders 
what the skimmed milk would be like. 

V. Practical Usefulness of the Data 

In order to make the results of practical use the writer has con- 
structed the following lists, which contain those words known to 
at least 90 per cent of the pupils in each of the grades tested. A 
textbook writer would probably be safe in using these words with- 
out explanation. Naturally, he would use many others, partly be- 
cause they would be necessary to express his ideas and partly 
because only through^ the reading of new words will a child’s vo- 
cabulary be increased. The writer would like to suggest, however, 

*A ctaituficadon of the curves appears at the end of Appendix D. 
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that words showing a lower d^ree of mastery than those listed 
should be introduced with some kind of explanation as to thetr 
meaning. Similarly, teachers could use the lists in their teaching, 
being especially careful to explain the words that do not occur on 
the lists below. In gener^, not only this list but the detailed list 
showing percentages may1>e used as guides to what can be expected 
of children at different levels. 


TABLE VI: Wordi Known by Ninew Per Cent or More of the Pupils in Each 

Grade Tested 


Grade 6* 

Grade 8 

Grade 10 

Grade 12 

agriculture 

agriculture 

agriculture 

abolitionist 

law 

bankrupt 

alien 

adjourn 

submarine 

boundary 

alliance 

administration 

town 

century 

amendment 

agriculture 

voyage 

coinage 

anti>slavery 

alien 


committee 

aristocrat 

alliance 


communication 

assessment 

allies 


community 


ambassador 


conference 

ballot 

amendment 


congress 

bankrupt 

annex 


continent 

bond 

anti’slavery 


court 

boundary 

budget 

appoint 

aristocrat 


defensive 

article 


discover 

candidate 

assembly 



capitol 

assessment 


expedition 

census 

authorixe 


export 

century 

coinage 

ballot 


fleet 

ctdony 

bankrupt 


freedom 

commerce 

bond 



committee 

boundary 


goods 

communication 

bribery 


community 

budget 


House of Repre- 

conference 

cabinet 


sentatives 

congress 


import 

irngation 

continent 

court 

campaign 

candidate 


credit 

capitol 



currency 

census 


law 


centuiT 


liberty 

decision 

civil 


defensive 

coinage 


majority 

democrat 

colony 


‘Tliete wen no Mch woidf ia Gndc 4. 
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TABLE VI: Wordi Known by Ninety Per Cent or More of the Pupils in Each 

Grade Tested— Conrifni^^ 


Grade 6* Grade 8 

Grade 10 

Grade 12 

manufacture 

discover 

commerce 

merchandise 

document 

committee 

missionary 


communication 

mob 

emigration 

community 

nationality 

empire 

competition 

employee 

compromise 

naval 

expiedition 

conciliate 


export 

conference 

offensive 

finance 

congress 

patriotism 

constitution 

peasant 

fleet 

continent 

pirate 

freedom 

council 

political party 

goods 

court 

population 

credit 

senate 

greenback 


settlement 

House of Repre* 

creed 

slave 

sentatives 

crime 

stoclct 

immigration 

crusade 

strike 

currency 

submarine 

import 

current 

surrender 

inaugurate 

inhabitants 

decision 

ux 

invention 

defensive 

territory 

investment 

delegate 

town 

irrigation 

democracy 

transportation 


democrat 

treasury 

labor 

discover 

treaty 

law 

doctrine 

liberty 

document 

veteran 


draft 

vice-president 

majority 

duty 

violation 

manufacture 


volunteer 

market 

election 

voyage 

massacre 

emigration 


merchandise 

empire 


mtssionaiy 

employee 


mob 

enemy 

execution 


nationality 

exile 


native 

expedition 


naturalization 

natural 

export 


resources 

finance 


luval 

fleet 


navigation 

neutrality 

foreim 

fraud 



freedom 


*TlMn <*nn m Midi nordt in Gnule 4, 
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TABLE VI: Wordi Known ^ Ninety Per Cent or More of the Pupils in 

Grade Tested — Centintuil 


Grade 6' Grade 8 Grade 10 

Grade 12 

offensive 


official 

goods 

opponent 

governor 


greenback 

parliament 


patriotism 

hostile 

peasant 

House of Repreien- 

pirate 

tatives 

polls 


population 

illegal 

prohibition 

immigration 

prosperity 

impeach 

public opinion 

import 


inaugurate 

republican 

industry 


inhabitants 

senate 

internal 

session 

invention 

settlement 

investment 

slave 

irrigation 

standard of 


living 

jury 

stocks 

justice 

strike 


submarine 

labor 

surrender 

law 


legislative 

tax 

liberty 

territory 


town 

majority 

trade 

manufacture 

transportation 

market 

treason 

massacre 

treasury 

merchandise 

treaty 

migration 


military 

unanimous 

mint 

union 

missionary 


mob 

verdict 


veteran 

nation 

vice-president 

nationality 

violation 

native 

volunteer 

naturalization 

voyage 

natural resources 


naval 


navigation 


neutrality 

^Tbere were no svch wordi in Grade 4. 
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TABLE VI: Words Known by Nme^r P«r Crat or Mon of the Pupib in Each 

Grade Tested— Co«A'Rs/d 


Grade 6* Grade 8 Grade 10 Grade 12 


nullify 

offensive 

official 

opponent 

parliafnent 

patriotism 

peasant 

pioneer 

pirate 

political party 
polls 

population 

primary 

proclamation 

prohibition 

proposal 

prosperity 

protective 

province 

public opinion 

raw material 
reparations 
republic 
republican 

senate 

session 

settlement 

slave 

smu^le 

stan»rd of livmi 

state 

stocks 

strategic 

strike 

submarine 

suffrage 

supreme court 

surrender 

tax 

territory 

testimony 

town 

— • ■ ■ I— A—" ■■ » I -- 

*Thm irete w fu^ «e(4t in Ondt 4 , 
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TABLE VI: Words Known by Ninety Per Cent or More of the PupSi in Ei^ 

Grade Tested— CenfiMird 


Grade 6* Grade 8 Grade 10 Grade 12 

trade 

transportation 

treason 

treasury 

treaor 

unanimous 

union 

verdict 

veteran 

veto 

vice-president 

violation 

volunteer 

voyage 

witness 


‘There were no luch wordi in Grade 4. 


There are in existence a number of frequency counts of the 
special vocabulary in various textbooks for use in elementary 
school, high school, or college. Anyone interested in so doing can 
check one or more of these counts — which show the actual num- 
ber of appearances of each technical word in a particular book — 
against the percentage knowing each term, or against the level at 
which each term is really learned. The writer will present one such 
analysis and compare it with the results, for the purpose of showing 
how this practical use of the data might be accomplished. The 
words occurring in the highest fifth of the frequency distribution* 
in a textbook widely used in Grades XI and XII were selected for 
study. Although this book is recommended chiefly for the final 
review of American History in high school, it is occasionally used as 

*These words were in the highest 6fth of the distribution table, but are IM 
one-fifth of the total number of words. They were select^ by ttkin^ enttre 
variation of {frequency shown on the table-^rom over 300 to 1— dividing OM 
distance by 5, and then copying down those words that had a frequency irpiii 
300 to 240— the highest fifth. Over half the words had a toal frequency of eitto 
1 or 2; the words selected constitute only 16 per cent of the total number of words 
—which was 862. 
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km as die eighth grade, and with fair frequency in Grades IX or X, 
in case the subject is taught in those grades. There were 137 words 
selected as above described. Of these 137, 22 were not included in 
the tests (one would not expect perfect agreement between the 
words in a list based merely on frequency in a single book and 
words based on analysis of several books, plus newspaper counts, 
plus judgment of teachers), 3 were used as examples, and 2 ap* 
peared in unsatisfactory items, thus leaving a total of 110 words 
for which results were obtained on the tests. The percentage of 
children in Grade XII recognizing these 1 10 words is shown in the 
table below. 


TABLE VII: Percenta« of Twelfth Grade Pupils Recognizing Each of 110 Words 
that Occurred in the Highest Fifth of the Frequency Distribution for a Single 
Textbook 


Per Cents 

100 

95-99.... 
90-94.... 
85-«9,... 
80-84.... 
75-79.... 
70-74.... 
65-69.... 
60-64.... 
55-59.... 
50-54.... 
45-49.... 
40-44.... 
35-39. . . . 
30-34.... 
25-29.... 
20-24..*.. 
15-19. . . . 
10-14.... 
5-9.... 
0-4.... 

Total. 


Number of Words Known 

12 

11 

5 

2 

1 

3 

1 

3 


no 


Seventy-two words (or 66 per cent) were known by 90 per cent or 
more of the jphildren, but 38 (or 34 per cent) were known by less 
tlian that j^portion. Teh words (or 9 per cent) were recognized by 
less drrat dtrte-fburths, but more than one-half, of these high school 

[ 180 ] 





practical usefulness of the data 

seniors. There was a total of 862 words found in this text; it is 
reasonable to suppose that the mastery of the remaining 72S, which 
had frequencies somewhere below the upper fifth of the distribution 
table, would be appreciably less. It would seem, then, that at the 
end of high school these pupils were not entirely ready to read texts 
using even the most commonly appearing words. If this text were 
used in Grade VIII, there would be only 22 of the 110 words studied 
known to 90 per cent or more of the children; for Grade X there 
would be 53 such words. Although both frequency counts and ob- 
jective tests have their limitations as educational techniques, this 
comparison between them seems to leave no doubt as to the lack of 
foundation in the matter of vocabulary upon which this textbook 
rests — even at the grade for which it was intended. 

A similar study was next made showing twelfth grade mastery of 
the 133 words found in the highest fifth of the frequency distribu- 
tion in the author's own counts of special vocabulary in newspapers 

TABLE VIII: Percentage of Twelfth Grade Pupilt Recognizing Each of 101 Wordi 
that Occurred in the Highest Fifth of the Frequency Distribution of Words 
Found in Newspapers and Current Magazines 


Per Cent 

100 

95-99... 
90-94... 
85-89. . . 
80-84... 
75-79... 
70 - 74 . . . 
65-69... 
60-64... 
5S-S9. . . 
50-54. . . 
45-49. . . 
40-44... 
35-39... 
30-34... 
25-29... 
20-24..: 
15-19... 
10-14... 
5-9... 
0-4... 


Number of Words 

9 

10 
5 
3 
2 
5 
3 
3 


Total. . 


* • 


101 
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aiid current mafSitneB. Of these 133 worde, 26 did not appear in 
the tcft (moat of them being too easy for luch inclueion)» 5 were 
used as esamplcs* and 1 appeared in an unsatisfactory item. The 
remaining 101 showed degrees of understanding by twelfth grade 
pupils as shown on Table VIII. 

0( these 101 terms for which test results were obtained, 61 (or 
60 per cent) were recognized by 90 per cent or more of the pupils; 
24 were known by some proportion, between 90 and 75 per cent; 
the remaining 16 were known by less than 75, but more than 50, 
per cent. This showing is not quite so good as that for the text- 
book results just presented. It suggests that the high school grad- 
uates are not adequately prepared to read the current materials 
through which they can most readily follow contemporary history. 
There was a total of 1,010 other words found with lower frequencies 
in the same newspapers; these would presumably show less ade- 
quate mastery. There seems no escaping from the fact that the 
vocabulary problem is far from being solved and that preparation 
of assignments in history or reading of the newspapers is likely to 
continue in sUUus quo until more meanings are acquired by the 
pupils. 

It is the writer's hope that others will find the data of value in 
other and equally interesting comparisons. The sole object in col- 
lecting the detailed information presented in this paper is to have it 
ustd. 

VI. Conclusions and Constructive'Suggestions 

It remains to point out certain relevant lines along which further 
work needs to be done if the problem of vocabulary is to be ade- 
quately solved. 

(1) There needs to be further study of the words revealed by 
word counts so as to get them classified in such a way as to be 
directly useful to teachers. The writer would suggest a division 
of all the special words found into "essential” and "accessory” 
lists (on the joint basis of frequency, importance, and sociological 
value) and a further <^a8sification into levels of difficulty on the 
basis these and further test results (plus, preferably, a logical 
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grouping within each level). Such work would eventuate in lists 
for ead) grade, or at least each level in the school system, so that 
the teacher could readily see (a) what words pupils at a given stage 
of development can reasonably be expected to know and (b) 
of these words which ars the more fundamental, and should be 
taught first. Such lists would have a practicability (and would 
therefore get into actual usage in the classroom) that could never 
be achieved by a long, alphabetical list — ^which would, in all prob- 
ability, be used almost exclusively for research purposes. 

(2) Much further study needs to be made of the validity — not 
the reliability, which is known to be high — of objective test items. 
Most people seem either to assume that words correctly recognized 
in such a test will automatically be known if met in another con- 
text, or else take an equally extreme position and state that the 
objective test tells nothing whatever about recognition in any 
other situation. Presumably neither group is right — and in any 
case both must be merely rationalizing a point of view since nobody 
knows the truth. It is the writer’s guess that the objective test 
reflects actual knowledge that is transferred to reading situations 
more often than not; but only extensive and impartial investiga- 
tion can settle this question. 

(3) Since school children obviously need to be taught the special 
vocabulary of history by some more adequate means than are now 
employed, the writer would like to suggest that texts should be 
worked out — certainly for grades below the college level — with 
careful study as to the introduction of new words. Thus it might 
be possible to introduce one new word on every other page, or 
something of the sort, with careful explanation of its meaning. In 
connection with the word lists above mentioned, the texts might 
become media for special training in vocabulary as well as for 
transmission of historical information. A simply-worded glossary 
at the back of texts, with footnote references for difficult or new 
words, would be a most welcome addition. In any event, some 
technique needs to be worked out so that texts will not dump 
a mass of unexplained — and hence largely unassimilated — words 
upon pupils. 


[ 183 ] 



TESTS AND MBASUKSMENT8 


(4) A consideration of the frequency counts hrom almost any 
text in history leaves one with die feeling that there is an enormous 
and unnecessary vocabulary burden in the form of words used 
only once or at most twice in the entire book. Such words are 
usually skipped by the pupil and, of course, such infrequent use 
does not permit of any real mastery. Nor can pupib be expected in 
one year to look up and learn the meanings of a considerable pro- 
portion of 862 words — to use the text mentioned in the previous 
secdon — and at the same time acquire a year's worth of history. 
Much of the dispersion of vocabulary in current texts seems to ^ 
due to efforts at style by the persons who write the textbooks. 
The writer would venture a guess that the style is largely wasted 
on the public school child and serves chiefly to add elements of 
confusion in the form of almost literally hundreds of special and 
difficult words used too infrequently to permit learning and with 
such profusion as to discourage ail but the most persistent. If the 
vocabulary could be concentrated by using fewer words and using 
them oftener, there would be sufficient practice given to promote 
real mastery of this more restricted, but more fundamental, vocabu- 
lary. Naturally, such concentration would be correlated with lists 
of essentials to be acquired at the successive levels. Such a con- 
centration of vocabulary has already been tried out with success in 
the construction of reading materials in the elementary grades for 
the mastery of general reading vocabulary. Each word of the most 
commonly used thousand (or two thousand or three thousand, ac- 
cording to the. grade for which the book is being written) is re- 
peated in the fundamental readers for a given grade in proportion 
to both its frequency of appearance in general reading matter and 
its difficulty. Learning to read becomes, with materials of this 
sort, a controlled process rather than one that proceeds on the 
basis of mere hope and courage. If history texts for the public 
schools could be rewritten upon such a basis, learning to read them 
wmild be not only easier — i^ich is, perhaps, beside the point — 
but infinitely more 8uc<^fui. 

(5) Naturally, one does not wish diildren to acquire words for 
die sake merely of knowing mtne words, but in order that they 
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CONCLUSIONS AND CONSTNUCTIVS SUGGESTIONS 

■* 

may read and think more efficiently. The writer has encountered 
a number of people who are inclined to scoff at any study of vo> 
cabulary on the basis that such study stresses “mere words.** 
Quite true. But no one can either read or think without them. 
Word meanings are the raw materials of constructive thinking, and 
no pupil is going to think* accurately without an adequate vocabu- 
lary. Anyone doubting the importance of “mere words” in the 
preparation of assignments in history should sit down beside a pro- 
bation student, watch him as he fumbles around with his reading, 
and investigate his thinking. After one finds out that he thinks 
“anarchy” was the outstanding form of government in ancient 
Greece, “reciprocity” is a form of revenue, and a “budget” is a 
statement of party principles, one begins to get a new sense of the 
necessity of mastering elementary meanings before advancing to 
higher realms of thought. 

Further light is shed on the essential need for adequate word 
knowledge if one will have patience with the stumbling probationer, 
will teach him the words he should have learned long before, and 
will then watch him gain markedly in his thinking — now that he 
has something with which to think. The writer would like to stress 
in closing, the strategic importance of the vocabulary problem, 
especially in the social sciences, because the words there learned 
are directly transferable to any reading of current events, and to 
urge that constructive steps be taken to bring about a learning 
that will act as an adequate basis for the development of far better 
reading and chinking than could be done with a vocabulary no 
more adequate than that revealed as typical by the results of the 
present investigation.* 

*The writer has just completed the preparation of two Soil vocabulary test 
forms of 140 scored items each, covering the essential terms. Since the entire 
learning curve for each question was available, the forms have heen constructed 
by painng the items according to the shape of the entire curve- Thus, two items 
showing percenuges passed in the grades as follows: 



Grade 4 

Grade 6 

Grade 8 

Grade 10 

Grade 12 

Item {!)... 

14 

21 

49 

58 

72 

Item (2) 

17 

23 

50 

54 

75 
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CCWCLUtlOiri AMD CONtnUCTlTB tUGOttnOMS 

« 

APPENDIX A: Woidc S e l ec t e d u the Eeeentiil Ooft ef Voetbulaqr 
IB Htttofy — CoHiimwd 


C. 


^convention 

*federaliat 


primary 

deadlock 

polidcal party 

coneervadve 

suffrage 

delegate 

ptogreaiive 

•pardean 

vote 

nonututte 

opponent 

repubUcan 
aocialitt < 

radical 


pTink 

pla^rm 

polidee 

deket 

Economic Terms: 

•whig 

lobbying 
patrwiage 
ipoila lyttem 


butineaa 

manufacture 

employee 

inflation 

conunerce 

itrerchandise 

Employer 

inveetment 

commodity 

production 

later 

market 

company 

property 

strike 

panic 

eompeddon 

raw matedal 

union 

speculation 

coniumer 

rebate 


etocks 

exploit 

^shipping 

#bank 

communicadon 

export 

trade 

bankrupt 

^factory 

corporadon 

tend 

public udlitiee 

gcxxla 

capital 

transportadoo 

import 

induitry 

monopoly 

credit 

prospedty 

wealth 

truet 

criaii 

depreciadon 

finance 


D. Socialofittl Ttrwu: 
arietocret rural 

peacant urban 

ilave 


faoaety 

conununitT 

homeittaa 

pioneer 

planudon 

aetdemcnt 


£. Letal Ttrmi: 
*atmttaty 
Slegal 
%T^id 
'ce 


centua 

inhabiunu 

population 

negro 

race 

mob 

riot 

natjonalior 

nadve 

naturalisation 

Appeal 


lightt 


fcaae 


uncoanttutional Iconvict 
crime 

alien deciaion 



emigration 

cxpanaioti 

immigration 

migration 


l^ucadon 

l^inatitution 

invention 

reform 


emancipation 

freedom 

irulependencc 

liberty 

oppreaaion 


^people 
^private 
^public 
public opinion 
atandard of living 


jury 

teadmony 

verdirt 

violation 

witneaa 

OKUt 

jurisdiction 


fraud 

graft 

eonapira^r 

ioaurrectKMi 

rebdiion 

revolt 

revolitdon 

eeceerion 


^Ut qratbel k aHd to indieBU tkow voida that the bbUmt 
MMM« k, eaak from a Kit of «M«tiak. 

#rkk »Mbel b wed to UicaU thoM ipaads that VHV, for « 
in tha taA, alttaeek tWb oMmtial Aa r acUr b ddbr. 


vonid DOW, on tbe barir af foftlM* 
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APPENDIX B 


A 


TB8T OP CONCEPTS TOED IN THE SOCIAL STUDIES 

Ay L. C Pimn, T)m Ohio SUto Hatwtlty 


Name.. 


Place. 


« 1. Who U tbt bMd of tho UniUd SUtM govornmontT (a) tbo Oovornor (b) Um 
S ocroUry (e) tho Prcikdont (d) tho lUnf 

« 2. Wbat Daaw ii glvoa to land naxt to tba aaat (a) tba moantaliia fb> tha ooaat 
(c) tbo plaina (d) tbo Talteya 

a S. What Is tbs daughtor of a king caUodf (a) a gnoon (b) a yHneo (e) a gantt^ 
man (d) a prlnctss 

• 4. What name is given to a place where thoosands of people live? (a) a boose (b) a 

city (e) a village (d) a school 

* 6. Where would you eapect to find a judge whan ha is at work? (a) In a e oiutr o o m 

(b) in a store (c) in a schoolroom (d) on a farm 

e What name is given to additions to tho constitution? (a) dootrlnea (h) amend* 
manta (c) assemblies (d) declarations 

7. Which trord means loyalty to one's country t (a) neutrality <b) migration 

(c) putriotitm <d) constitution 

8. What is tha law>maklng body of ovary state in tha United States called? (a) pur* 
liasMnt (b) c o ngress (e) judiciary (d) iagislature 

9. In which form of government do the people have the graatest power? (a) an 
abeolnte monarchy (b) a rapublie (c) an oligarchy (d) an embassy 

10. What term is need in referring to the entire system of courts? (a) legislative 

(b) administrative (c) supervision (d) judicial 

11. Whatarepeople when they first enter a foreign country? (a) citlseas (b) laborers 

(c) aliens (d) peasants 

12. Which group contains tho oflicial advisers to tbo president? (a) Bouse of Repre* 
sentativaa (b) Senate <c) Supremo Court (d) Cabinet 

15. What is one important purpose of politics? <a) to eliminate graft (b) to elect tha 
beat possible man (c) to organize public opinion (d) to keep j^wAid ^ toe from 
being aeUah 

li. What is a primary? (a) a method of choosing the man to be voted on (b) a poUtf* 
cdT convention (e) a group of foreign oAeials (d) a Uw dealing with capital 

IB. What fa a ‘tieket’’? (a) the candidate e^ has been nominated (b) a precldaifa 
hctaa elaetlon (e) a Ifat of poople to be voted on (d> an unwritten law 

16. Whidi weed lefeta to the affaire relating to one’s own cotnitryT (a) foreign 
(b) bib i ninfl e n nl (c) d e meetie (d) d^lematfa 

17. Whntfaesiadtbythe'fauthorttlei'’? (a) Ugh olkiala (b) f««eifaan (e) heads 
of cerpontloas (d) lueeutein 


* USED AS EXAMPLES 
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TF.CTS AND MEASUREMENTS 


SR wittt Sow Mflnt* MnT U) U nff« pwiiilwnirt (b) to m<»i witf»crttoi 
(«) Um liflit to f«to (4) • m«Im 4 «4 uRtttii t imiR 
IR Wfetah an fociaa af mtani n ao m tnt (a) faelortaa (b) fgnato (c) nniTw 
(4) adbaob 

IR Ta whMk 4o« totoruattonal niattoaa nf«T (a) nialton witUa a eaunliy (b) 
nlattena bi t waw i a ooontrr aa4 Ha poaaMaioM (e) nIattoM b at wa aa taro 4cpni4' 
aat atotM (4) rolatloaa b a twaaa aa tadapaadaat toata aa4 a fareltB cooatry 
IL Wkat la aa ladiiMaal eallad wbaa ba has aaaoaaaaS Ua latontiaa of ranntag for a 
aartafat anaoT (a) aa oppmant (b) a 4a)ocate (e) a caiwHdaU (d> an oAcer 
IR froia wbat an praeadanta oftan 4aHn4T (a) Iradttioa (b) aabtaet (e) taa»- 
paratan (4) eampatgaa 

IR WUeb word aiaaBa to gin anotbar poraoa tba powaa to do aomathiagt (a) anlorca 
(b) rafoaa (a) aothoriaa (d) aacowaignleato 
li, WMdi word iBoau oppoaad to alanryt <a) pro-alanrp <b) anto-banom (e) 
pae-Attatteaa (d) aati^Torr 

IR Wbaa doaa a paUtiaal aan^toifo taka plaaat (a) Joat baton a aaiabni of congraaa 

(b) joat baton a war (c) Juat baton aa alaetioa (d) Hwt baton a law ia paaaad 
IR What ia a fadaral gowaaiaBtf <a> a anioB af atotaa (b) a eoaatitaUoii (c) a 

roaba (d) a proriaea 

IT. WUeb raUgioB ia chiafijr oppoaad to CatboUeiaaT (a) Ht ad oi am (b) Atbalam 
(a) ProtoaUatUm (d) Coaaaiaalaa 

IR Wbi^ an porta of tba cmtitatloBT (a) artklaa (b) tnaUaa (a) ntooi (d) 
oardleti 

n. Wbat doaa oao aatioa ao to a Ha aaa auko to aaothart (a) % eottaaoditp (b) a 
aaaana (e) a aooatltatioa (d) apropooal 

IR Of wbat la a inmn tba boadt (a) a eoaatrp (b) a eitp <e) a atato (d) a 
towB 

n. Wbat ia anotbar aama for a atatoT (a) a onanoswaaltb (b) a eongroaa (c) a 
dapartmant (d) a dtp 

tt. Wkat ia a plank T (a) a ballot (b) a pdHieal alqgaD (a) aa itua ia tba platfona 
<d) a doadloak in tba partjr 

M. What ward aiaana aineh tba bobm aa fnadaaaf (a) aon nlga ty <b) tpraaap 

(c) pacHam (d) Ubertp 

14 WUdi an onaetadt (a) Tardkta (b) dabta (e) lawa (d) dlapatebaa 
IR WUeb ia a narnltT (a) a profltoar (b) a banker wbo loaaa laaiwp to tba aaamp 
(a) a aaiwlT-onliatod addar (4) a eriniliial wbo ia awaiting aioaattoB 
IR What ia tba ebiaf bnalnoaa of the anaantin d^artaMat of tba gooanuaoat? ia) to 
abeUab km (b) to maka lawa (c) to into rpao t kwa (4) to mfona km 

IT. WUdi word ia moot aimllar in m oa n ing to daapotkat (a) anardip (b) tpranap 
(e) ooa fa dantinn (d) naka 

IR 9p what pneoaa ia doth auda from woolT (a) mUag (b) fanning (c) dpabw 
(4) maaafnataiiag 

M. What kanollm auM far ‘‘goada”! (a) mr i b a w d i ai (b> bnaknaa (a) laetodea 
(diatna 

4R WUdi t kr aaa nfan to tb^ aaUn baaiaiaa of awkim woalae aktoimt W tba 
waoka abagn <b) tike wooiaa iBdaatrp (a) tba wea l g ro w a ia (4) tba iVrif 
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ACTUAL TESTS BLANKS 


4L Wtet hawm mms SoMdalal (a) K Ufoemm kM vahiabk (b) B i»« 
ba7 BOT* <«> It Lh to go back to Um ailat (d) Ik out ba aiad la fwratga « a — t ri al 

41 Wbo to Um lAdal fawd at a aoBU^t (a) kb« <b) pnridaBt (a) dtoMkot 
<4) Jo^a 

41 WhkbtoaiawButoriair (a) etotb (b) wood <o) papor (d) obalr 

44. Wbat to tho puxpoao at dlrioanopt (a) to eanp oa affatra wttUa a oeoaitoy (b) to 
oleek gavormaoBtal oOeoia (e) to eoadaek rolakkaa bakwooa eoaatriao (d) to 
toeraaoo fadaotria] ootpak 

41 What Bamo to given to kblaga broai^ into tho Unltod Statoa (ram a foniga oa a a 
trpT (a) oatoma <b) tmpofta (e) ptonona (d) bonds 

41 What dou roligtons toloraaeo aio a n t (a) sukiag all poapio botoag to aaa ebarob 
(b) belioTing ovorjihlag In tho Biblo (e) boltovtag la setoaoo iaatead at tho ^arto 
(d) allowing people to boUevo what tbop wQl 

47. What to tho naato given to tho activittoo of any gvonp (or tho patpooa at getting a 
eortaia bOl throagh tbs togtolaturef (a) lobbying (b) oloetlonoorlng (OX vattag 
(d) secret servieo 

41 What were tho people eallod who wished to do away with slavtiyT (a) probdiittea* 
tots (h) saffn^ttota (c) abolitiontots (d) moBarchiats 

49. What to meant by logtolattoaT (a) the pamlng of laws (b) tho eonvlottng o( 
criminals (e) tho eoUactlng of money (d) tho oatoielag ai laws 

50. What happens when a tow to millUtodT (a) it is onsetod (b) It Is ratUtod (e) Ik 
la no longer In force (d) it boeomos a doereo 

51. Which word means those who ontar a eonatry to live theroT (a) 

(b> Immigrants (e) patriots (d) bnsinass bmb 

SI Whkh word means giving money to an oficar at the govommont to Inflaaneo hto 
aettonsT (a) bribery (b) counterfeiting (e) piracy fd) pelltks 

SI Whkh toars refer to tho mannfseturs, sale, or transpertatWof llqaorT (a) sedl* 
tioB tows (b) natarnlisatlon tows (c) pr<dtlbitlon tows (d) corporation laws 

SI Which word moans a prolonged attempt to conquer some fertiged ptoosY (a) a 
battle (b) a siege (c) a aUrmish (d) a retreat 

81 Which means a robber OB tho sou 7 (a) bandit <b) thief (c) pirato <d) higb- 
wayman 

61 Wbat to one way of attacklBg a (ortillca^T (a) by sklrmtohlng (b) by — 
sacie (c) by retoforeemeats (d) by bombardment 

87. What to a eoort*s doctoien caltodT (a) a tariff (b) a verdict (e) a nronnssl 
(d) a proUbtthm 

81 Whidi word means to gain someone’s good win after it hu «"*e been tostf (a) 
tolerate (b) soeese (c) conciliate (d) condemn 

H Whkh to Om tow-maUng body of a dtyT (a) eeancQ (b) aaaedatton <e) eon* 
greu (d) sonate 

tl- What word moans bringing goods into a eonatry wtthont paying dnty 7 (n) 

(b) graft (e) eoantarfetoiBg <d) amnggUag 

•L Whkh word meaaa to give each pacsoa whatovar ho dautwisY (a) eharito (b) 
d i vec u (e> dupottom (d) Jaatka 

<1 Which woedusau a s(<9pfiig of war loageaoagb to dtocBM peaesf (s> dkanaa* 
maat (b) insarraetion (c) treaty (d) armtotke 

41 Whkh to a result of iadepandonea, in a go ver n m an tal senu? (a) enUmltad frsadam 
<b) aatt-tonmauBt (e) eoatrat by akhaa (d) kek of htMativa 
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TESl'S AND MEASUIlfcMENTS 


14. WiMt M « pttm owiMd br AaotW ealMt («) u Mpmr (b) « «mif* (e> • 
prttrt (4) A aim 

M. Which to Ml la vanttoiiT (a) pnfaiavto (b) c t aa inh oa t (e) Ufhtaiat (4) 

M. To what to a panto moat oftait 4aaT (a) too maeh capilal (b) too UtUa baaknptep 

(c) toe anidi apaeslatton <4) too Uttta tn4a 

ty. What to tbopmaaa of ouTTfac tUnta froBOBO plaea toaaethar calladT (a) trana- 
latlon (b) tranolttoii (e) tfanaaetloa (4) tramportation 

M. What 4o pott caat at an aloetiont (a) a polkp (h) an ofltolal (e) a promtoo 
(4) a haDot 

W. What aaroo to froqoonttp flMB to a now town in a saw c o nn tr y T (a) aottlomoDt 
<h) metropolto (e) prairie (4) atoca 

yo. What 4oaa a poreon do when he azploita otherat (a) ha to a dtplomat (b) ho 
eharfM too nnch for what ha aoUa (e) ho toada a riot (4) he him othen to worit 
for him 

TL What word meana an afraeffiont between two eoButriaa to five each other trade 
prIvilegoaT (a) rociprocltp <h) emancipation (c) finanea (d) eonsplraep 

n. Whtoh tarn meana legal pmitohment bp death? (a) aaaaaaiitatioB (b) enaentloa 
(fl) app (d) imprtoonmont 

n.. Whan to thera nanallp a period of roeonatraetlon T (a) after a period of proaperitp 
(b) aftor a war (c) after a religiouc pemacntion (4) after a new praeideat to 
atoeted 

74. Tfhat done “Jortodiction*' mean? (a) to obtp (b) to refnae lopaltp to (c) to 
dafond (d) to hoTO power o*ar 

71. Which name to fleen to the head of a mintotrpt (a) premier (b) rovemor («) 
aonanl (d) arabaleador 

74. Which of the fallowing may be deacribed aa an event? (a) intereet in edacation 
(h) movement toward etricter law enforcement (c) development of an indnatrp 

(d) doelaration of war 

77. TThat to the money which the government coilecU for public nee called T (a) 
reTanne (b) appropriation (c) currencp (c) allotment 

7». TEhtoh beioage to the navpT (a) tank (b) regiment (e> cavalry (d) cmtoer 

7». ffhieh to the llnal act In electing a preaidentT (a) voting (b) petiUon (c) com- 
promtoe (d) nom^tion 

•D. T^hkh to a political party? (a) Puritan (b) aopnbUcaii (e) Eptoeopal (d) 
Qontlto 

n. Which acttoa docee porta to navigation? (a) naval battle (b) embargo fe) cMl 
war (d)aviatian iw 

•t WUA la a ■tatament of leUgloaa baltofi? (a) creed (b) petition (c) leeoluUon 
(4) tohatp 

n. Wbkh to a oltlaaa-anap celled oat aaly la tima of amarganey? (a) volaataaia 
(b> mOltia (e) brigada (d) poUoa 

N. Whtoh pel U toa l party to oi^caad to private owaatahip? (a) Aati-Saloon Laana 
<h) r e ea la toaniat (e) loyaltot ^(d) eoctoltot 

?? *?."•*** ODaenatkMB aaUad? <a) commieeionata 

(h) ieli grt ii <e)ptoltai WdamocnM 
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TBST OP CONCEPTS USED IN THE SOCIAL STUDIES 

ky L. C. Putw, Th« Ohio Stela Uatvanttgr 


Name Aga Grade 

School — — Place. Data 


1. When two eonntrici make an acnemant to help each other what la it ealMt 

(a) competition (b) diaaimament (e) alliance (d) election 

а. If fifty people rota unanimoualy tor a certain ofllcar, how many ootM did ha fatt 

(a) 26 (b) 49 (e) 0 (d) 60 

2. What la one way of gettinc aoldienT (a) by conflacation (b) by amat (e) by 
draft (d) by penalon 

4. Which term deacrlbaa property that ropreaenta a large money valoe T (a) p or afty 

(b) wealth (c) profit (d) rapply 

б. With whet ie finance chiefiy concerned? (a) fanne (b) money (e) paopla 
(d) rlghta 

6. Which ia the adence that la concerned particularly with aUpa (a) rnDHariam 

(b) chemiatry <e) agrlcolture (d) naeigatlon 

7. What ia meant by reclamation? (a) cirflixing a aavage (b) edoeating a child 
(r) converting a baathen (d) making a waate land uaable 

8. Which word meana an ofllclal written ctatementT (a) empire (b) doenmant (c) 
conference (d) oommlttee 

9. Which if the oppoaite of a raaitary oeenpationT (a) arnmnalticn (b) evaeoatieB 

(c) proacription (d) rehabilitation 

10. Which word refera to thoaa people who era employed in miUa, factortea, or aUaaal 
(a) monopoly (b) wealth (c) eapitaliam (d) labor 

11. What ia given to a people to prove their ownerahlp of the land they cattle vpeaT 

(a) petitioD (b) charter (e) doctrine (d) amendment 

U. WUdi people are hoetfle? (a) citixena <b) employaaa (e) enemlea (d) alUaa 

18. What doe* eaeeaaion mean? (a) to combine (b) to cooperate (c) to withdraw 

(d) to captme 

14. Bow long ie a centory? (a) 10 yoaia (b) 60 year* (c) 100 yean (d) 1000 jaan 

16. Which word meana the period daring which a kgialatar* la mootlagT (a) ditet 

(b) vodi (c) aea e i o B (d) congieae 

Id. WUeh ia a form of enrreney T (a) a ponnd of angar (b) a eoOaga dagne (a) a 
im i aawm (d) a aUver ddlar 

17. What am yaople pemanently located in a country eaUadl (a) (h| 

lemitBcra (e) emigraBta (d) capi t a H a t e 
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TBrr» AND MEASUREMENTS 


IR WM* ft filM wMt M • Dm mf memm ftiwiMw 

(4) fT f S t ir * **“**"^ r “— <•) rifTT^fr- 


IS. 


Wtet mRm It takn on t tMtr if Ikwt watfaf tp p iu it 
<k)itlt4iMht4 («) h it nrtMtd (4) » it ■■ttiii 


Itt <t) R it ' 


H. Wktt atat to fifta t« tht chtof kttotoltot b«4F «< I^m 4T (t) Etrltoaart 
(b)tM(nH it) tMtf gtntrtl (4> MidMtac 

n Wbo tt w n4i t Enaidtnt 4yiic ia «AmT <a) tht Mcntarr c4 ataM (t). ttoa 
tEtahar ef fha Hooaa (e) <ha aUtf jaattoa M tta eapin eaort (4) tia atoa* 


tt> WMeh ward to aMat »<"««■»' in maanMc to tnatt (a) mmtt^a (b) aatton (a> 
jaattoa (4) eovporatioB 

It. Who ava bolUcaMBtar (a) tha naotral eonntrtoa (b) tha aaontetoo tba4 an 
4afaata«t (a) tha eevntrtoa that aria (4) tba eoaatrlaa Uwt an Stbttac 

Mi WhSdi worI aaaaa f na d o m fna olaaotyT (a) pafdoa (b> acHattoa (a) aaua* 
aipattoa (4) aaaaipttoB 

M WUdiword Biaaaa prtTilofaa BO ana can totally taka Inn not (a) lawa (b) dae* 
triaaa (a) rlthta (4) la a Ututiua a 

M Who mmj ba hnpaafhadt (a) a bnaiaaaa aua (b) a eriaiiaal (a) a npartar 
(4) a pnoMaat 

IT. Whtohanaaltodelartjrt (a) miatotan (b) a t a t aaM o a (e)fo«anan idljodgaa 

M Whtoh waa4 OHoaa fraadon Iroaa diatariwaoat (a) tnaty (b) E*Ma (a) boo> 
ffltty (4> law 

M On what ia than a tariff T (a) thlata laanafactiirad ia a ooaatry <b) thlata aaot 
ant of a oanatry (a) tUago famatad ia a aoantry (4) thiaga aaat tato a euuatiy 


Ml Whtoh to a way of UUag cartaia Eoblto ofleoat (a) apEltoatton (b) raaigaattoB 
(a) oadetatBaBt (4) oppolataMat 


SL What to a poUT <a) a pUoa to aolo (b) a boBot (a) oa oafalr a i a rttoai (4) a 
typo of atohnwa 

n, WUd woa4 to oixaUar in maaaing to tanalidt (a> doUa g naat (b) uMMoUtirtiontl 
(a) fagftloa (4) taaaaoral 

n. What datarmlnoo tha iiauaaaltlaa, eoarforta, and hixartoa wUdt tha amga honai 
*—***■ ^mbT &) poUto optatoa (b) Matao' righto (a) ataadard of Uriag (4) 
aahanal aaffraga 


M WUdi wood nfan to poUttoal oamq^toaT <a) graft (b) lyaddng («) nnlt 
(4) aratiBy 

M. What to tha amiay aaltod that oaa aonatry bonvara trona aaothar (a) debt (b) a 
dlyidawrt <e) a praadoai (4) a traaty 


ttoapanoaait^aAI (a) whan ha to atok (b) 
a aoa 



Fttt hand af tha Jwdfetory bnaeh'af tba 

(a) capital pototohaaaat <d) tha 


(e) 

not hit awB (4) whoa ha haa baaa to pttooa 
af Bto g a lt (a) iafnattoo <b) I fli ar ty (a) 


(a) pohlto optotow 
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ACTUAL TESTS BLANKS 


M. Wkiit m thoM pMvto aJM m* IMmb te ^ «Matiy thtf wn kws h? 
U) cMUua (b) eMnH (c) Ammimm (S) tikm 

SOl Wtet k « eoiMamqrt (a) ■ Stem (b) a Imbm <c) ■ tnatr (S) a p«IMm 

4L Wbat do re« can a »on «r Iom pirnuniwt ■o ltl iwia t is a oowkerT <a) a 
doaweracy (b) a eoloBj (c) a tariff (d) a conttB a trt 

41 Wbat to ropadiatadT <a) d^to (b) gawtaor o (e) Boortgagoa <d) laad 

41 Wbat to a caacaa to? (a) to alaet a caadtdata (b> to doioat a eaadMala (a) ta 
doddo opoa a eaadkUto (d) to paatob aa uawtUfag eaadtdato 

44 Of what bodtoo to Coagnao eoapoaadt (a) ParUaaaiit aad A i aa mb ty (b) Priaa 
Mlatotor aad Cabinat (e) Hobm of XaproaantaUTaa and Sonata (d) Haaaa of 
Lords aad Hoosa of Commons 

45. WbJeb to tho aama of tha armp, in AaMrioan btotorp, that fooght for tha aouUMni 
statasT (a) Engitob (b) eoafadarato (a) nalon (d) eolralal 

41 What to tho titio of a mambar of tha bigbor braaeh of tho Congroas of tho Uattad 
StataaT (a) raprsooBtattTo (b) aanator (a) dslagata (d) Judgo 

4T. Wbieh to aaotbor word for eommareot (a) g o t a n u aaat (b) trada (a) raUgloB 
(d) arlmo 

48. What aro tho rlowa of poopU tn goaaral oaBodf (a) trado nalona (b) pablia 
opiakm (a) statas* rights (d) aMl oarotoo 

4P. What doaa a eompaay seU whmi It wants to tecroaaa Ha aapital T (a) aalartoa (b) 
drafts (a) loss (d) atoebs 

H What doss latarstats maaaT (a) wtthin a atato (b) batwoan elttos (a) bstwoan 
statas (d) within a eonatry 

IL What to tho oSlatol eoronKmy that marks tha bogtanlag of tho ProsldoBt's tarm 
callodT (a) neminstlon (b) eoronatim (a) co B To n ttoa (d) tnangwatioB 

51 What do w« aall tho piaao whoro dtoagrsomonts aro triad boforo a jadgoT (a) 
aapital (b) nary (a) aonrt <d) pablia opinion 

51 What happens whan monay to inflatadT (a) it will bay mors (b) It to not poaaibla 
to fanrest it (a) it can bo nsod to pay intamattonal dobu (d) H will buy lota 

51 Whiab to tha Sat or rolling typo of aountry? (a) soashoro (b) valley <s) prairto 
(d) dMdo 

66. Of what to a "grant” oftsn oompooadT (a) laad (b) poopio (a) labor (d) 
offlcan 

51 What nama to givaa to an area of partly aattlad land belonging to a eonntryt (a) 
provineo (b) natkm (a) rapablie (d) i n vo a tme Bt 

57. What does maiority moan? (a) a ndaority (b) aa oStoor in tho army <e) moaa 
than Sfty par oant of tha votas (d) a sneeaasfnl mndidata 

51 What naaso to gtona to tha Frasldanfs tana of oStoaT (a) dseada (b) pasta (a) 
oampaiga (d) adaaiatotmtian 

58. Wbat naam to givan to tho money or goods paid by one eonntry to aaothar as aattio* 
nMnt for dunago done durin g a war? (a) tartt (b) c u rre n cy (e) npaiattoaa 
(d) loan 

SI To wUdi does naval rofar? (a) (b) daaarta (c) tratna (d) taanala 
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TESTS AND MEASUREMENTS 


SL By «lMt mUmS Is «hs gvfstMs's sOss flM7 (s) by toMsthis (b) by sIscttoB 
(e)byi«asB <S) by sBdsmMst 

O. Wbs SBSHplii tbs ysysqrf <a) tts pops (b) flw prssidsat (e) tbs prfaas mia* 
Islir (4> lbs psHss 

SI. WbsS an tbs ptcfpht calM srbs Jola aa amy of tbslr own aaosrST (a) ysa ws ls 

(b) allaas (e) sapttaUsts (S) nfanrtsan 

4L Wbs Ssskhs arbstbsr or ast a aaut Is goOty of ssorSsrr (a) tbs Jaiy (b) tbs 
jsdt* (e) tbs w l t s iisii (S) tbs lawysn 

SB. Wbat to bmSs to tbs aUatT (a) saaSy (b) moasy (e) dotbss (S) baeoa 

SS. What to tbs ootar sdgo •< a dviUssS am saDsSt (a) aistiopolto (b) sobatbs 
(e) frsBttor (d) ssacaaat 

ST. Of wbon to tbs popslattoa composodf <a) baaks (b) psopla (e) dsbto (d) 
proposals 

SR Wbaa to aa lafoBatloa aiost ofton ossd? (a) during a striks (b) dnriog a battto 

(c) daring aa szploration (d) during a royags 

Si, mwt to a tariff wUeb ssaki to hslp homo iadustriosT (a) tariff for isscnno only 
(b) tatoraatlonal tariff (e) fm trado (d) protsetiTs tariff 

TR Wbat to tim opposlts of ‘^irbaa”t <a) fonign (b) ntotropolltaa (c) OUtorato 

(d) rural 

n. Whtoh wood moans opon rsrolt against autborityt (a) diplomacy (h) agrienltnrs 

(e) dshata (d) insurroctlon 

7R Whtoh rstors to tbs lowest social class la many cou n tri es T (a) roprsaa ntatlv ss 
(b) psasants (c) Inbabitaats (d) dolsgatss 

TR Wbat ara nations on tbs sasM aids la a war caUodT (a) offleors (b) alltoa (c) 
marshals (<l) driUans 

fR Whtob to a mobT (a) a Soot of batttosbipo (b) a dsersase in prices (c) an 
snltad group of paoplo Id) ■ torn on imports 

VR Whtob to a form of conspiin^ against me's eountryt (a) sodition (b) opposition 
(e) erittoism (d) immlgratloa 

TR WbsB a law to npaalsd, what to always the multT (a) It to anforced soon rigidly 
(b) It to a m o n Ssd (c) it to no loager in loros (d) it to a soares of dissatisfaction 

TT. ^ shows when one cou n try atopadifd anoOnr baglnsT (a) 

n. Which la BomdbtBg to bo obeysdt (a) law (b) (c) report (d) 

oslAntiaa 

1R WbM tons Bssaao tbs doafatg of tbs porta cl ono ee mm by tbs nasal fnees of 
aaotlMrt (a) rndmego (b) ami battle (e) daetotrs fW blockade 

SR Bow Sosa a person biesnii a candi d ate for ofieeT (a) by iaSsat (b) by nemtna- 
tton (c) by alaeti^ to offtoe (d> by appointment 
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TEST OF CONCEPTS USED IN THE SOCIAL STUDIES 

Dtvumi 6y L. C PUMKY. 11»« OUo Sut« Uniwatty 


Name - - - A^e Orad«. 

School — Place.. Date - 


^ 1. Who of the following repreaenta the United Statea officially in a foraifn country T 

(a) an ambaaaador (b) a aenator (c) a patriot (d) a yovamor 

* 2. What if a froup ot people carrying on a particular buiinoaa called T (a) a maa* 

date (b) a company (c) dividenda (d) a community 

* 3. What ia likely to reiult when all partiea in tha government have a 4 )ual atronctllT 

(a) imperialiam (b) maaaacre (c) amendment (d) deadlock 

* Where did the rich people of the South live in the early dayaT (a) in homaateada 

(b) in log cabina (c) on plantationa (d) on river boata 

* 5. What word meana about the aame aa revolution? (a) extradition (b) adminiatra* 

tion (cl penalty (d) rebellion 

6. IVhat ia neceaaary if deeert land ia to become fruitful? (a) migration (b) rapara* 
tiona (c) irrigation (d) capital 

7. To what doea agriculture refer? (a) ftahing (b) mining (c) farming (d) manu- 
facturing 

8. What ia the oppoaite of "modem'’? (a) mediaeval (b) colonial (c) ancient 
(d) foreign 

9. Which ia an expreaaion of the govemment’a opinion on aomething Important? 
(a) doctrine (bl empire (c) budget (d) commerce 

10. Which term refers to a religious war? (a) an cxpanalon (b) n revolution (e) n 
crusade (d) an offensive 

11. What term may be uaed to describe the peaceful settlement of diaagraimanU? 
(a) arbitration (b) nnoaution (c) nullification (d) proclnmation 

12. What ia the nation’a capital T (a) the largest city (b) the chief aeaport (c) tbs 
seat of its gavemmcnt (d) the center of its wealth 

13. How long la a docade? (a) 10 years (b) 23 years (e) (0 yaars (d) 100 yasi* 

14. When do current events take place? (a) a long time ngo (b) n abort tima ago 

(c) in the future (d) in the present 

15. Whkh name is given te the countries ruled by an emperor? (a> state (b) alliaaca 
(c) commonwealth (d) empire 

10. What kind of people abdicate? (a) traitora (b) employees (e) kings (d) alavaa 
17. Which la a denomination? (a) Baptist (b) Jew (c) (Jenum (d) Negye 

•USED AS EXAMPLES 
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TESTS AND MEASUREMENTS 


tiL WkM S* tv* MtiMi Imm to il ftof VMt 4 «ito Sliw* tUsft hat 4o wt 
vaattosatowahaallhMrSWMHit (a) nrtM <h) raittjr 
<S)toaWiia» 

UL What an thaSMltataal«M asaatrjr with aaMhwaalMT (a) anndMato (h) 
i n a H ai iiti <e)natotBthM (4) aagottotlna 

M. Whkh a^JoaraiT (a) a tnatj (b> toa Uat (e) oe ag ma (4> tha a i ai a tH aHn 

tL WMdi la aa old aaldlart (a) aa laoalM (h) a aUaiam (c) a oatoraa <d) a 
aaaolat 

ta. What k aaaallT tha objaet of aa axpadKtoat (a) to ianat aoBMthlac (b) to 
aeatnt aaoMthlac (e) to Rlaa taantoiat <d) to d li eam aaaMtUaff 

111 Taaaid irhieh la tha aoaaamtioa aMnaaat dtaaetodT (a) toward fdhieal ear* 
tastloa (b) toward boalaaaa tafatm (a) toward rdtglaaa dtapatoa <d) toward 
praM^HiB wwn 

ti. mUk la a aoattoaatT (a) Paaasa (b) Loadea (a) Aala (d) frman 

■b What aaaaUjr dotaraiaaa yoor nattaaalUyt <a) tha paapla 700 Un whh (b) tha 
aahaal foa attaad (a) tha aaoatiy yaa wara bam in (d) tha fonraaMat of yaar 
aHy 

M. Bow doaa tha aaoy gat aMoay to baOd a now ahipt (a) by anaohnant j(b) by 
agycoyrlatlaB <a) by aegalaitton (d) by tarnation 

IT. Whiah b a anthad of daeUUag how nncb tax ahaold bo paid by dtafaraat paapla T 
(a) aaaaaiiaant (b) andor a aaaant (a) adbaraaMmt (d) awnnbaant 

II. What b aaothar nama for tha Unitod Statos gooununantf (a) tha Coafadaraay 
(h) tha Uataw (a) tha Praaldant <d) tha Conforanea 

II. Ib tha aarty daya of tha Unitod Stataa wbleta party faoorad a atrong oontral goram' 
aaatT (a) Tniaa (b) Xnow*nothinga (a) Fadanb (d) Damocrato 

M. Whan b tha yavamnant'a nonay kaptt (a) In tha Booao (b) in tha Senato (a) 
fathaailat (d) in tha traaniy 

IL What doaa a eity ardinarOy aall whan It n aad a monay for aoma pnUb fanproTaanontT 
(a) amdlta <b) bada (a) dMdanda (d) bonda 

SI. Wham an anatoBoa oOeara toondT (a) in eitba of antry (b) in atato ca plt al a 

(a) in awaa t ahopa (d) in tha ton largaat aitbo 

W. Whiah word oMano a oMthod of taUlimeing poopb to agroo artth yooT (a) mratt 

(b) propaganda '*<a) olaetioB (d) rktory 

M. What b patting agataiat oaob own ao an try callodT (a) anrdor (h) witcherait 
(a) rbt (d) traaaon 

K. ynUk b a anaQ political dhrbioa for beal gonm n wntt (a) aaetba (b) rogbw 
(a) aoanty (d) toltury 

M. Whadda yaa do whan yoa banpaarawn eoaatoy to aatbo toanatharr (a) aad* 
gtato (hlntnat (a) aamador (d) abdieato 

IT. Whbh b aiandt (a) rardH (b) crodita <e) bad (d) aortmUty 

M. What doth tin Praaldant aw a at fa naa appoint whan ha wanto aoaw aaattar h waa U - 
jatodT <a)aeainpaBy (blagaahaaay (a) a conadBaia (d)adalagata 

M. Whil gina tha paopb tha *TtandaB»aBtal bw” by which tiny an gonmadT (a) 
aanatlMlen (b) BBaaifoato (a) aan inhi y (d) pattiaBaaat ' 


[ 198 ] 



ACTUAL TESTS BLANKS 


40. BTMdiltBlBwT (•)apatltla« (L) • tepcteh (4) ■ iiowt 

4L Wkat ii ■ kliw ta mrt i m m aIMT (•) rt»iia1kr (b) pilM MtetoMr (•) pral> 
tet (d)wf>ntga 

41. WUdi party wmt ^poad to b«loarf dortag tbo BmtatioaHy WwT <•) WMg 

(b) PrognalTo (e) Abotttinam (d) Tory 

4L Wbot It 0 ptnoD alkd who loata In a oMottltd aetla of bit own aartryT 
(o) aBigaat (b) ploMor ' (c) tUa (d) proprioter 

44. Wbat BtBt it flan te tbo orgaaiatioit of omplopoaT (o) Mttoa (b) polttloal 
poitia (c) pioMta (d) onlat 

45. Wbot word maa t pita whoa poopit UaT (a) a moMpatr (b) a aomosltjr 
(e) a eoart (d) a dtadt 

40. Wbat aa affata within a coantry attodt (a) latrietod Cb) Baaaeia] ' (e) ataim- 
factaa (d) intanal 

4T. Wbat it a dvil waif (a) a war duriag tbo pat 8S yoaa (b) a war terddltc tor- 
oral atioa (e) a war botw ao two Mtioa (d) a war witbla a tiaglo eoaatry 

4S. What aa tbo tt ro a g a t atloa lii tht world oftoa caMadT (a) oaplra (b) loacao 
of MtioM <c) powta (d) donoeata 

40. What it tha oppaito of offooaWaf (a) aaonratia (b) aotia (t) iiaontla 
<d) dofoa i a 

60. Which word it mat timilar ia aiaiiiiit to portodt (a) poar (b) atto a (e) 
aoath (d) oa 

61. BHtot happont iHiob tboa it a dnawdal crialtT (a> a now proald i t it olootod 

(b) prlcot aa lowored (e) thoa it a war (d) tboa b a poriod of pat par i ty 

6t. WhatbaatoT (a) a oomprombo <b) a pbUthaJ party (•) a datMod far fairft 
(d) a rofntai by tho paaldont 

SK Wbat b tho tax pot opoa thhiri bronght into a eonatryt (a) baatB (b) daty 

(c) dbeonat (d) itteomo 

64. Which word lofoa to tbiagt doot by toldioar (a) military (b) aavbl (c) oMl 

(d) aoro n a u tical 

66. Wbat b tha act of ealUiig o«t ttoopt caHadT (a) mobObatioB (b) eoB0naatlaii 
(e> indietmaat (d) raeantation 

6K Which b a writtaD bwT (a) a aogoUatioa (b) a aUtaU (e) a po bi ica t ioo 
(d) a Toto 

57. If yaa and tomaoao olae both want to bo oteetad ProoMaat, wbat da yao oaB tbo 
otbar poratmT (a) opponent (b) ally (c) ofco pioaldoat (d) dtaaoc^ 

68. Wbat b tho largoat group of pocmb living nndor tbo tamo gona mm a n tt (a) dty 
(b) nation (e) coonty (d) prorinoo 

68. Wbat b tho noiilt of a raatrictiaaT (a) you lafnoo to do ao m at hfai g (b) yaa ara 
pravootad from dataig aomothing (e) yoa an anco a i a gad to braak a law (d> yaa 
an pat in JaH 

00. Which name b g i ro n to tha mbaat of patroaagoT (a) dril oorviea <b) tpoila 
ayatom (c) factoay ayatem (d) aociot Barriea 

0L Wbat poltey b o pp oaad to that of oont nlit a Hoa in gonenmantT (a> atatoa* rigbiti 
(b) aaavMitloa (e) public ^bdon (d> magaa ebarta 
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TESTS AND MEASUREMENTS 


M. What io w* «0 Uw point at atom at a peraon otraiclf In fnvor at om nlda of an 
vgnaMatT (n) fanniM (b) dwninntiny (e) oietoiion (d| putteMi 

M. Whkli word inpliM tlM oppoaito of prtmitivo f (a) onrijr (b>eJ«flisod (c) nadont 

(d) ortctwU 

M. What Is (hpoa by a railroad to oortaia ooaipaaloa ao tliay can aaad thlaga awrt 
ahaaply Uwa otb« cempanlaBt (a) r aaol n tioa (b) labata (c) ballot (d) earn* 
patitJon 

IS. What doos a Imkar buy la Um “BMrlMt"? <a) baaha <b) aUtaka (c) aMrehaadiaa 
<d) labor 

M. Wkkh of Um foUowlac words appitas to tbo pmetloo of lowatiag of prieaa by oaa 
company so as to gat biuinoaa away from anotharT (a) rsbatUon (b) oilgratloa 

(e) output (d) compatltion 

S7. Which word moans to sand goods out of the country in arblch thay ars prodneadt 

(a) retail (b) export (e) aall at wboksala (d) import 

M. What is the axeiusive right or privily of ongagiag in a particular bnainaas catledf 

(a) compatition (b) monopoly (e) capital (d) rsasarch 

19. WUeh is a method by which a govamraeat raises aumsy T (a) robataa (b) diplo- 
macy (c) taxes (d) salarim 

Tib Who of the following go to foreign lands to teach religion (a) eoavarta (b) mla* 
slonariaB (c) ambassadors (d) bishops 

71. Which is a commodity ? (a) wheat (b) people (c) scaaery (d) adneatioa 

fX. What If attuily iawtadt (a) a mortgage (b/ aamt maoey (t) a parsoa 
soBM goods 

Tt. What aaam is given to a parson who givts toatiraoay befoeo a ccertT (a) witaaaa 

(b) lawyer (c) Judge (d) Juror 

74 To what places does Fan'Amoricaa refer? (a) U. 8. and (b) America 

and Europe (c) Nortti and South America (d) the U. S. and tu possaBBioas 

78. Which word means any eoauaereial or industrial Badartaking? (a) baaklag (b) 
businasa (c) exchange (d) contract 

TV. What is the condition whaa therd is no goveramaat? (a) Slarsry (b) freedom 

(c) axpansioa (d) anarchy 

77. Which word means that a country has taken over eentrol of a MaalkT country with- 
out Utf latlar’s eoasant? (a) imperialiam (b) daamcracy (c) anarchy (d) 
patriotto 

78. WU^ is a diplomatic ageat? (a) a prsaident (b) a Jnry (c) a aiiaistar (d) a 
priaat 

79. What ii a groiq* of ahips uader oaa coau aand salladT (a) flsut (h) earavaa (e) 
frigata (d) army 

84 Whidi word d me rib s s any countoy in ediieh a penoa b not a dtiaea? (a) foraign 
(b) aattva (e) doaaaatic (d) eontlaontal 
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TEST OF CONCEPTS USED IN THE SOCIAL STUDIES 

Dtmtd iy L. C. Funn, T)m Ohio SUto Unlvtnitjr 


Name _ _ Ag« Grade 


SchooL. — _ Place __ Date 


1, What naiM wu yivtn to larye araaa bafor« th«y bocama atataaT (a) oltiaa 

(b) factorial (e) tarrltoriai (d) Juaclaa 

t. What ia a bndfatt (a) a caah aeeonat (b) a lut of namaa (c) an aatiaata of 
azpanditona (d) a payroll 

8. What happona whan a law ia put into affect T (a) It la laflalatad (b) it to dacraad 

(c) it to aancttonad (d) it to tnfottad 

4. What la the nama of a formal arraemant batwaan two or more conntrtoaT (a) a 
treaty (b) an odiet (e) a raaolution (d) a atatuta 

6. Which to a Jounay on tha oeaanT (a) caravan (b) eonaplracy (e) voyafo <d) 
traaaon 

6. Which boat can traval undar tha water? (a) crutoara (b) draadnaofht (e) iroe* 
clad <d) anbtnarina 

7. Who ara the "aoldian of tha aaa”? (a) tha aallora (b) the militia (e) the 
marinea (d) tha population 

8. Which word maana aomo dtohonaat act? (a) abdication (b) prohibition (c) auf* 
fraga (d) fraud 

9. Which to a war of anraaoioBT (a) a lockout (b) an Invaaion <e) a rotraat (4) 
a battia 

10. Which adjeettourafera to tha country in which one to bom? (a) altou (b) forolffB 
(e) adopted (d) uatiru 

11. Which to a puhlie atataaMUt? (a) prodamatlon (b) Imparialtom <e) eipoeltioB 

(d) aaaaaddy 

U. Whidi ward Bwatia to ftvu approval? (a) repeal (b) aanction (c) yule (d) 
revolt 

18. What to meant by democracy? (a) ffovemment by a king (b) govommont by a 
dictator (c) government by the people (d) goremment by the nobility 

14. When an army baa loet a war what doee it do? <a) it diea (b) it aurrandan 
(a) it eonquera (d) it manauveta 

18. What g roup of poeplo rank thamaalvaa higheet in aoeiaty? (a) middleman (b) 
ailatocmta (c) pioneora <d) democrata 

10. What doaa a eanana uaually count? (a) reeo u rce a (b) capital (c) aap en dl t m ua 
(d) peoplo 
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TSm AMD MMAMUDSMEim 


IT. WkM 4tm altnllM mmmi (»} • Mattjr (k) » Mad «f kv (c) a 

Imairtaf firm plm to plm id) • Csm td toto to 

If. WhMi tarat auv rate t« a «tt3da a frpattmrt at tta fomamatt (a) 

honaa (V> wart («> i ia iw ii (d) aaMaot 

II. Wtet k a pka te otwral batfka aalUdf (a) akga (b) aary (c) 
(d)a9aaaka 

IB. Whm a goww u aa nt triw to laqarow aaea t kaa, vbat k ft ealkd? <a) rofamdos 
(b) (aMatiw (e> ratona (d) oaawaifa 

IL Wbat anot yw baw ia ardor to b orro w aioaoyT (a) bOk (b) laaryarr (e) cradH 
id) oorporatiaa 

II. WUeb of tbo fdlawinc k a OMtbod of ooamaakatioat (a) to roota wa t (b) toIo> 
pboM (o) prodaetloo (d) poUtlei 

11. Wbl«b of tbo following tutoaionto boot dooorlbw a banknipt eoontryT (a) it boa 
too away poopk tai it (b) it baa no agrlraltaro (c) it eaaaot pay tto dobta (d) H 
baa loot ita cotonka 

t/L Wbleb of tha foQowtiic k a kgkktiva aaaomUyt (a) ompbro (b) e e o fraai (a) 
world ooort (d) buoao of odneatlon 

M. Row do kbotora bhoI ottaa attempt to fono an om^oyor to rako tbeir wogwT 
<a) by talBff to war (b) by loing to aaotbar factory (c) by goinf to a fortign 
oo ua tr y <d) by going oa ateika 

M. Wbat dow it aMOB to bo natarallMd? (a) to go to aaotbar eoantry (b) to bold 
aa odko k a gowonuaaat (c) to boeonte a eit kaa of a foroiga eoantry <d) to 
boeoma a roiy woalthy paraon 

tl, Wbat BOBW k gWoB to a aawU orgaakod groap of poraona wltb eortala work to 
dof (a) a coaunlttoo (b) an aaaomUago <e) a porliamont (d) a coagroaa 

■, Whid aro '^bUe otUttko'*? (a) gaa oomponka (b) atato b a nk a (e) cotton 
aUDa (d) iron foaadrloa 

II. Wbat Barn k ghron to tbo vonoy aaod for flnanrlng tndostryt (a) tax (b) tiaao> 
ory (a) ragiaw (d) oapitol 

N. Wbat k tneaay paid oat by tba gorammant calkdT (a) agpondltaroa (b) tazao 
(o) tailSb <d) aaaoaaauBta 

li. To wbat oapraaaioB dow tha term cotoagt rafarT (a) tha waoring of cloth (b) tba 
raking of wkwt (c) tba mining of awtala (d) tha BBaUag of piaew of monay 

IB. Wbat k tba anway callad wUdi tbo kwr of a war poya w damagw to tbo winnar 
(a) kiB (b) ap^oprktion (e) mortgago (d) k d ematty 

n. Wbkli ora aiaaltioBBT <a) aoMloia (b) naiforma (c) traaaporta (d) obalk 

M. Whkh tom nfan to dtiwT (a) aatkoal (b> iatantoto (e) mnaidpal (d) 
pnotaikl 

N. WhatkaplatlotmT (a) a Ikt of oaaAdatw (b) a atatowont of prtodplw (e) a 
NUgkH eroad (d) a pcdltkal akgaa 

M. Wba|kapalltka? (a) a reply (b)anfb8al (elacopaal (d)araqnoat 

IT. ki wi^ patkd aroro b a tM w earrkd on hg knlgbta in annorT (a) tha ealonkl 
gartad (b) tba piwhktorlc po r kd (e) tbo madiairal p e ri od (d) tho rarohitioii* 
ary l«kd 
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ACTUAL TESTS KLANItS 


SI. Wtat tana wp la to i th* poaitlaa «( « eoaatnr Im aot teka cUm li « w 
batWM* two otter coaotrlMT (o) AoMarotk (k) howit («) portUl (4) aow 
tnl 

M- ^VU^^umten of tte HoomT (o) lo l o fot w (k) oonoton («) mofon (4) 

4ft. Vkieh woi4 la oaod to daae^ aooM tepeitaat giaklf tkat aao4a to ko aottlodT 

(a) aa ianu (k) a dodaioB' (e) aa odiet (4) a aaaatioo 

41. Wkat la tte oppoaita of coa ao rratlraT (a) laeial (k) radlaal (e) poUUeal (4) 
Okgal 

42. Vkieh of tte foDowtat adjoetivoa boot doaerikaa a law that la haaad aot m aay food 
roaaoaabutonparaoeallikaaordlallkaoT (a) doBMeiatk (k) Utafal (e> foroiga 

(d) artitnry 

45. Vhea a lofialatnro kaa votad for ooBMtkiag wkat b it eaUadT (a) aa idoa <k) an 
act (c) a ertma <d) a propoaal 

44. What dooi violatioa BMaaf (a> braakiag a law (b) atfaias a tiaatjr (e) alael* 
lag a eandlilata (d) dadariag a war 

46. Whldi eanally tea tte tmallaat nanbcr of poopit la ItT (a) aatloa (b) atata 

(e) city (d) town 

41. Which word noana tte kUUng of a largo aombar of paoplat (a) aa oaeonatar 

(b) aa aeoaomy (e) a daeioa (d) a aaaaaert 

47. Which word daacribaa a eondltloB la which paopla ten aora aeaoy lhaa aanalT 

(a) pomty (b) proaparlty (e) prohlbltloB <d) protaetioa 

48. To what doaa tte word ‘‘■Tooabaek'* rafar? (a) a dlriaioa of aa amy (k) a kind 
of ahip (e) a kind of nonoy (d) a dapartaaiat of tte traaanry 

40. What ia tte naar of gooda eallodt (a)4te coaaaaugr (b) tte raforawr (e) tte 
traitor (d> tte prodaoar 

60. Which ara coatrabaad daring a warT (a) hooaao (b) gana (e) b a ttla a (4) 

61. What do aatlona do whan tbay diaagrao, bat want to And aoaw frlaadly way oat 
of thair dimealtioaT (a) daelara war (k) bold a oonfaronao (a) got oat propa* 
gaada (fl) olact aaw oilleiala 

62. Which ia a rapraoantatiTa of tte goTanaaaat la a fonIgB aoaatryt (a) aonsai 

(b) aoToroiga (c) vieo-piaaidont (d) haakor 

68. How did tte early AawrieaBi atata thair tad ap a a df a eo from EnglaadT (a) by a 
riot (b) by a treaty (e) by a political party (d) by a daclarattei 

64. In hiitory what aaaw waa givea to a part of tte eonatry bofore It kaeaatt 

a atatat (a) repnblie (b) tvritory (c) atatota (d) aation 

66. What ia calladT (a) a frontiar (b) a domJaioa (e) a daapotiam (d) an 

o^iro 

64. What aaoM ia given to tte givlag np of amiaa and navioat (a> e o m pr o ndae (b) 
caneaa (c) apeeatathm (d) diaanaanwnt 

67. When a fort ia aapedally wen plaead what ia it called T (a) atratagle (b) celoiiial 
(e) ooaaarvative (d) patriotie 

68. By wbom are araminationa teld for tte filling of aaaay poaitlona in tte govemnaatt 
<a) party poUtiee (b) pabUe opiaioB (e) Laagaa of Nattona (4) ohFfl aarvfae 
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TESTS AND MEASUREMENT- 


». WMA -wri ■ to gkf M to wam» aflMr tmm m't M<— ? U) Mfsrta (b) bbI- 
lUjr (e) c Bii eiJt (d) mo 

Mi Wlwt BMM lo f im i to tho aMVUMBt of poeph frea tbo ooston porta of tho Uaitod 
StatoototlMwootorat (o) ■oo w o t a o (b)«xpoiMioo (e)eoM«rmlaa (djproeodoot 

tL Whot oro taw boodo of doportmata to tlw United Statoo fomrawnt colkdT (o) 
eoMttlo (b) pro a U n (c) ooerotartoo (d) Maoton 

M What do jroQ do whoa poo poM- a rtaolationT (a) roto (b) nnlUiy (c) 

(d) vtto 

M Which U tha aamt of tha honaaa Uaad to by aosM of tha aarly aotUara to tlw Mlddla 
WaatT (a) hotola (b) boaaatoada (c) plaatatloaa (d) boafalowa 

M Whan a (ronp of paopla pat vary aticry and ezeitad and want to abmr how tboy foal 
what aomaUaMi takaa placaT (a) aa appropriation (b) n riot (e) a caapalfn 
(d) Bonopoly 

M Which word rafara to aoma certain period of ttow to hiatoryt (a) politico (b> 
Bantrallty (c) rafioM (d) nuwaacra 

M Which word Indieatoa tho gOTenuBent’a attiuida on aoma important quootionf (a) 
policy (b) laaua (e) patronofa (d) criiU 

ST. MThat axlato whan ona croup of paopla are unfair and cmal to another fronpT 

(a) Indapaadan c a (b) tolaranea (e> traaaon (d) oppreacion 

M For what have paopla often baan paroacutod by the church T (a) migration (b) 
war (e) haraay (d) contraband 

M. Which party is at praaant oppoaad to tha Bopoblicana? (a) Democratic (b> 
Fadarallat (e) Liberal (d) Alwlitioniat 

TO. Which term rafara to tha procaaa of m^ng thlnga in an indnstry? (a) produetien 

(b) raw amtorial (e) wagoa (d) ra aw r c a a 

71. Which word manna aomathing to be rotad on? (a) bill (b) credit (e) diplomacy 
(d) proclamation 

n. Which party davelqted within tha Republican party during the pant twenty yaan? 
it) Whig (b) Domoenttc (e) Prograaalva (d) Sodaiiat 

Tt. Which word rafara to tho conditions to tha place wharo you lire? (a) religious 
(b) natmaal (c) national (d) local 

^4. Which tomris applied to a parson who bolds a poaitioa of authority in tha gevam- 
mantt (a) principal (b) lawyer (c) oSictol (d) aareant 

Tt. IThtdi to a raeeT (a) TnoA (b) Negro (c) Amarioan (d) Qnakar 

Tl. Whan a gOTammant wtahoa to pniibh pao^ what doao it edtan do with their prop- 
arlyT (a) defenda it (b) omAacatoa H (c) anbordtoatoa it (d) makoa it into 
torta 

77. WhiA word maaaa tha braaklng of the tow? (a) atrika (b) war (c) erimo (d) 

waaMt 

7S. Whatla an aaaamblyT (a) the alliaa to a war (b) a gioiqi of paofda (e) a Ugh 
tnrtf (d) a aat of towi 

tS. TThlei word m aa n a tha c oha titut ad anthorittaa of a nation? (a) tha banka (b) the 
praaident (c) tarn cavanunant (d) tha aapranm ooort 

M WhiA it a •‘coat to antol" oeantayt (a) Africa (h) Maodco (e) Canada (d) 
ftaam 
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APPENDIX C 


APPENDIX C: Dittribudom of Total Score* on Preliminary Form* B, C, and D 

of the Te*tt 


PRELIMINARY FORM B 


Grade* 

Score ' ■ 



4 

6 

8 

10 

12 

80 






78-79 



i 

8 

40 

76-77 



6 

13 

91 

74-75 



5 

29 

103 

72-73 



16 

53 

141 

70-71 



35 

62 

134 

68-69 



45 

66 

112 

66^7 


4 

65 

87 

105 

6f-65 


2 

68 

120 

69 

62-63 


1 

76 

78 

38 

60-61 


4 

111 

53 

41 

58-59 


4 

123 

55 

25 

56-57 

i 

8 

126 

58 

20 

54-55 


15 

91 

53 

14 

52-53 

3 

15 

99 

51 

8 

50-51 


19 

86 

40 

7 

48-t9 


23 

85 

24 

5 

46-47 

i 

27 

74 

12 

8 

44-45 

2 

27 

52 

12 

2 

42-43 


36 

49 

11 

2 

4(M1 

3 

44 

49 

7 

2 

38-39 

3 

34 

25 

4 

2 

36-37 

7 

44 

34 

6 

1 

34-35 

3 

47 

25 

6 

2 

32-33 

11 

31 

10 

5 

2 

30-31 

17 

22 

12 

3 

1 

28-29 

24 

19 

7 

6 

, , 

26-27 

31 

22 

10 


1 

24-25 

20 

14 

2 

2 


22-23 

22 

4 

1 

. , 


20-21 

13 

5 

5 

1 


18-19 

7 

3 

3 

1 


16-17 

.. • 11 

4 

1 



14-15 

10 

1 




12-13 

7 

2 




10-11 

3 

1 




8-9 

4 

1 




6-7 

3 

• . 




4-5 

1 

. 




2-3 

2 





0-1 

. . 

•• 


•• 


Total Caae* 

.. 209 

483 

1,397 

926 

976 

Median 

.. 26.1 

39.2 

55.5 

63.4 

70.3 
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TESTS ^ND MEASUREMENTS 

PRELIMINARY FORM C 


ScOM 



Grades 



4 

6 

8 

10 

12 

80* 

n-n 

76-77 

. 


• 


. ^ 

74-7S 



4 

6 

121 

72-73 

• 

i 

8 

32 

173 

70-71 

. 


8 

32 

140 

68-69 

• « • 


17 

46 

124 

66-67 

, , , 


23 

48 

117 

6f-6! 


1 

24 

61 

34 

62-63 


, . 

24 

68 

73 

60-61 

i 

2 

51 

37 

63 

58-59 

. , 

2 

61 

52 

32 

56-57 



70 

62 

s6 

54-55 


2 

70 

56 

37 

52-53 

2 

8 

93 

49 

28 

50-51 


5 

94 

SO 

21 

48-49 


16 

104 

44 

8 

46-47 

1 

12 

103 

33 

14 

44-45 


12 

139 

23 

6 

42-43 

2 

24 

111 

33 

S 

4(M1 


30 

99 

18 

38-39 

1 

26 

107 

19 

4 

36-37 

4 

25 

87 

10 

1 

34-35 


30" 

71 

8 

2 

32-33 

2 

46 

46 

4 

2 

30-31 

6 

49 

44 

8 


23-29 

7 

38 

22 

4 


26-27 

16 

36 

19 

5 


24-25 

13 

35 

12 

5 


22-23 

25 

29 

8 



20-21 

18 

12 

6 

2 


18-19 

12 

9 

4 

1 


16-17 

26 

6 


1 


14-15 

20 

8 




12-13 

12 

2 




10-11 

8 

2 




8-9 

12 

1 




6-7 

7 

1 




4- 5 

7 





2- 3 

4 





0- 1. 


•• 




Totkl Cun. — 

114 

460 

U34 

817 

1,100 

i- 

- ■■ 

— 

— 

1 „ 

- 

M«d»ii|. 

— -jfc 

18.3' 

32.0 

45. 9 

57.2 

68.2 


t th>t SMardMta|l»qriiMMcfdiiiibnM«*MiiMda*e)uavl«.i«vi«c 
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ATPBNDIX C 
PRELIMINARY FORM D 


Gndts 


ScoK 



4 

6 

8 

10 

13 

80 





• * 

78-79 



4 

11 

96 

7^-77 



6 

14 

108 

74-75 


, , 

11 

19 

98 

72-73 



18 

30 

81 

70-71 



36 

26 

71 

68^ 


1 

35 

46 

57 

66-67 


1 

61 

52 

42 

64-65 


3 

70 

33 

46 

62-63 


2 

85 

56 

32 

6(V-61 


2 

99 

45 

27 

58-59 


3 

106 

43 

29 

56-57 


6 

114 

42 

16 

54-55 

i 

13 

100 

39 

14 

52-53 


10 

111 

27 

11 

50-51 

1 

19 

117 

25 

7 

48-49 

1 

15 

115 

15 

8 

48-47 

3 

28 

89 

22 

8 

44-45 

, . 

31 

72 

10 

7 

42-43 

1 

36 

67 

9 

7 

40-41 

3 

29 

38 

6 

7 

38-39 

1 

20 

42 

1 

5 

36-37 

5 

29 

33 

2 

3 

34-35 

4 

30 

16 

2 

3 

32-33 

4 

33 

9 

2 

8 

30-31 

11 

33 

13 

2 

3 

28-29 

18 

31 

5 

3 

2 

26-27 

11 

19 

4 

• 

3 

24-25 

17 

18 

5 

1 

2 

22-23 

19 

7 

2 


2 

20-21 

17 

16 

- 


* e 

18-19 

24 

8 

• 


• • 

16-17 

16 

6 

1 


3 

14-15 

14 

3 

. . 


2 

12-13 

6 

1 

• . 


1 

10-11 

9 

1 

1 


1 

8-9 

2 

1 

1 



6-7 

5 

• ' 




4- 5 

3 

• • 

• • 



2-3 

4 

. * 

• • 



0- 1 

1 




- -1 

Total Cue* — 

201 

455 

L488 

583 

810 

Median 

22.0 

37,4 

54.0 

61.8 

71.4 
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TB8T8 AND MBASUftEMBNTS 


APPENDIX D: Percenate of Chfldtea to Each Grade Xecofnixiiig Etch of the 

Word* Tetted 


Word* beiof Defined 



Grade* 



Form 
and No. 
of Item 

4 

6 

8 

10 

12 

ambaatador 

(Used 

at example and not scored) 

C. 1 

authotidei 

21 

40 

81 

87 

94 

A, 17 

eoBiid 

19 

38 

61 

83 

87 

D, 52 

covemor 

61 

61 

84 

88 

97 

A. 30 

tin^ 

(Not counted because of error in scoring key) A, 42 

nrinifter 

11 

26 

32 

32 

51 

C, 78 

offictat 

28 

59 

78 

91 

94 

D, 74 

premier 

10 

30 

55 

57 

57 

A, 75 

primetninitter 

(Not included in test) 




police 

(Not induded in test) 




pieddent 

(Used as example and not scored) 


A. 1 

Kpreaentativc 

26 

S3 

62 

76 

82 

D. 39 

feffeuiy 

4 

21 

23 

30 

59 

D, 61 

■enatoT 

27 

49 

59 

60 

79 

D, 46 

sovereign 

32 

45 

54 

72 

86 

C, 41 

•tatesman 

(Not induded in test) 




vice-president 

41 

74 

96 

98 

99 

B, 21 

assembly 

39 

61 

85 

85 

91 

D. 78 

bntjrau 

10 

17 

26 

52 

64 

D, 18 

board 

(Not included in test) 




cabinet 

11 

17 

75 

84 

93 

A. 12 

Gommiuion 

25 

34 

20 

31 

62 

C, 38 

committee 

28 

80 

92 

98 

98 

D, 27 

conference 

26 

75 

90 

92 

92 

D. 51 

congress 

21 

62 

90 

94 

99 

B, 44 

cnontifi 

30 

43 

73 

86 

93 

K, 59 

depanment 

K 

21 

13 

30 

39 

D.6\ 

House of Eep 

21 

62 

90 

94 

99 

B. 44 

lea^ 

23 

32 

43 

71 

86 

B. 40 

legydature 

16 

18 

62 

69 

78 

A, 8 

parliament r 

7 

50 

83 

93 

93 

B, 20 

senate 

21 

62 

90 

94 

99 

B. 44 

sesabn 

29 

42 

78 

92 

98 

B. 15 

anardiy 

12 

25 

60 

71 

79 

C. 76 

*coininiMm«alth 

30 

38 

57 

79 

79 

A. 31 

communism..., 

(Not induded in test) 




confederacy 

23 

32 

43 

71 

86 

B. 40 

despotism 

10 

11 

28 

36 

48 

A, 37 

democracy 

28 

50 

80 

84 

95 

D. 13 


37 

88 

88 

97 

98 

C, IS 

fed^ 

50 

60 

66 

75 

84 

A. 26 

gevaniment 

MO 

25 

44 

51 

51 

D.79 


. . •mbol i* MiA to Mieato tkoM worti tho amkor ttooU oot ■*> ia • Um of 

tet apaa w hidi ant 4aa an avaSaUh 
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APPENDIX D 


APPENDIX D: 


Penentaie of QtOdm in E«ch Grtiic Recosniiing Each of Ae 
Wordt Temd— CMHaW 


Word* bcinc De&ncd 


'unpenalism . . 

monarchy 

republic 

■elf^ovemmcnt 
tynnny .... . 
union 


Grade! 


8 


10 


12 


Form 
and No. 
of Item 


(Test Item unsatisfactory) 

(Not counted because of error in scotina kerl 
S4 79 87 89 94 

37 42 76 76 8S 

10 11 28 36 48 

28 57 58 69 84 


0,77 
A. 42 
A, 9 
A, 63 
A, 37 
C. 38 


city 

colony 

country. . . 
county . . . 
*dominion . 
nation. . . 
province. . 
state . . . 
territory. . 
town ... . 


(Used as example and not 

scored) 


A, 4 

32 77 

89 

91 

96 

B. 41 

(Not included in test) 




25 36 

S3 

67 

72 

C, 35 

34 71 

81 

88 

89 

D, 55 

37 68 

82 

89 

91 

C, 58 

14 49 

78 

88 

94 

B, 56 

61 61 

84 

88 

97 

A, 30 

37 88 

96 

97 

98 

D, 1 

81 92 

97 

99 

99 

D. 45 


alliance 

arbitration. . . 
diplomacy. . . . 
foreign .... 
international . 
negotiation . . . 
neutrality .. 
pact. . . 
peace . . . 
power* . . 
recipro'-ity 
treaty .... 


18 

62 

74 

91 

99 

B, 1 

13 

23 

31 

68 

83 

C, 11 

13 

23 

47 

56 

78 

A, 44 

45 

62 

77 

86 

90 

C, 80 

13 

36 

25 

40 

60 

A, 20 

25 

28 

42 

69 

87 

C, 19 

21 

55 

84 

90 

94 

D, 38 

(Not included in 

test) 




43 

72 

77 

81 

85 

B, 28 

19 

30 

36 

56 

76 

C. 48 

18 

27 

27 

37 

62 

A, 71 

34 

71 

91 

92 

97 

0,i 


amendment 

4 

27 

article. 

18 

22 

charter . 

21 

58 

constitution. 

30 

40 


83 

90 

97 

A, 6 

69 

86 

96 

A. 28 

60 

65 

77 

B. 11 

68 

82 

92 

C. 39 


bOI 

. . 7 

11 

40 

IS 

\J7 

59 

U7 

81 

fj 

84 

U, 

D. 71 

declaration 

. . 34 

62 

73 

84 

84 

D,53 

decree 

28 

49 

39 

53 

79 

C, 40 

document 

4 

47 

84 

96 

98 

B, 8 

law 

.... 63 

90 

96 

97 

98 

B, 78 

legislation 

... 28 

41 

72 

79 

87 

A, 49 

measure 

(Not included in test) 




petition 

21 

33 

55 

74 

89 

D.36 


. "Tlu* «]F«kol i* 
Ml term, m opw 


to indicate tkae «otdt the anther wooM not new 
Mt data art tTaabbi*. 


tedndahialitiari 


[ 209 ] 



TBtTS AKD lIBASiniEMBNTS 


APPENDIX D: Pereentaie of Cbfldfca n Etch Grtic RBcog nLu in Etch of die 

W«dt Temi—CotHiniitd 


Wovd* banc Defined 



Gttdea 



Form 
and No. 
of Item 

4 

6 

8 

10 

12 

pcodtmttion 

. 17 

45 

£0 

85 

93 

D. 11 

pfopoetl 

. 17 

38 

69 

79 

91 

A, 29 

pfoiVMon 

(tport 

retolation 

(Not included in teat) 
(Not included in teat) 

. 11 29 39 

47 

70 

D. 62 

rMCriction 

. 16 

28 

50 

73 

68 

C, 59 

•tttute 

4 

15 

17 

52 

77 

C, 56 

tbolieh 

*tbdicttc 

(Not included in teat) 

12 20 27 

34 

54 

C, 16 

tdjoum 

23 

48 

72 

69 

96 

C, 20 

innex 

14 

29 

47 

65 

91 

C, 37 

tppcdnt 

14 

39 

42 

71 

91 

B, 30 

tuthorite 

11 

23 

60 

80 

92 

A. 23 

compramiie 

28 

46 

64 

88 

95 

C, 18 

ew^e 

20 

29 

54 

64 

79 

D, 59 

*conciliite 

30 

31 

56 

73 

91 

A. 58 

^oonfiicate 

21 

33 

51 

71 

78 

D, 76 

entct 

47 

47 

48 

56 

73 

A, 34 

eidorce 

39 

66 

88 

88 

88 

D,3 

pent 

unpetch 

(Not included 
18 29 

in test) 

60 

87 

96 

6, 26 

iniuturtte 

nidlify 

23 

24 

73 

90 

97 

B, 51 

26 

41 

62 

80 

90 

A, SO 

ndfy 

12 

15 

54 

73 

83 

B. 19 

repetl 

43 

50 

58 

66 

83 

B, 76 

^repudiate 

12 

13 

28 

42 

55 

B, 42 

*Btnction 

16 

28 

50 

73 

88 

D, 12 

veto 

23 

41 

61 

88 

96 

C, 52 

catecutive > 

20 

27 

63 

65 

65 

A. 36 

hMative 

23 

39 

82 

86 

93 

D, 24 

jodicttiy 

39 

49 

55 

71 

83 

A, 10 

tppiDpritdon 

19 

41 

47 

64 

85 

C, 26 

tatemnent 

23 

37 

66 

90 

95 

C. 27 

hudfec 

IS 

20 

76 

91 

96 

D. 2 

*oeintBe 

26 

62 

93 

97 

97 

D, 31 

cntrascy. . . • 

47 

51 

86 

94 

99 

B. 16 

cuttonu 

23 

46 

58 

73 

87 

C, 32 

ddht. 

27 

74 

78 

83 

83 

B, 35 

duty 

16 

27 

66 

80 

93 

C, 53 

expeaditurea 

U 

15 

43 

69 

62 

D. 30 

Vwbtck. 

20 

37 

83 

93 

93 

D, 46 


. "TUineiMif o^telBdMatBtiioMiwdttiwMidMrwwMnetaoiriMMtataliMaf 
iM 'irilScli tMt tua iM 
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APrSHDIX D 


APPENDIX D: Petenitiite of QtSdnit in Each Gnde Rooocntiiiic Etdt of dN 

Word* Tnted — Cniimaul 


Words being Defined 

*nunt 

protective 

revenue 

tariff 

ux 

treasury 

doctrine 

iuuc 

policy 

reservation. 

centralization 

civil 

civil service 

domestic 

internal 

interstate 

local 

municipal 

states lighu 

administration 

regime 

*capiml 

patriotism 

prohibition 

reconstruction 

referendum 

campaign 

candidate 

caucus 

convention 

deadlock 

ddegate 

nominate 

opponent 

platform 

politics 

ticket 




Grades 



Form 

w.enJ 

'4 

6 

8 

10 

12 

SM rtOe 

of Item 

32 

47 

72 

88 

97 

B, 65 

18 

31 

63 

74 

91 

B, 69 

8 

19 

48 

54 

71 

A, 77 

27 

33 

78 

82 

86 

B, 29 

56 

83 

96 

96 

98 

C, 69 

73 

84 

95 

95 

95 

C, 30 

13 

29 

62 

87 

92 

C, 9 

11 

13 

18 

30 

63 

D.40 

11 

24 

25 

39 

62 

D. 66 

(Not included 

in test) 




16 

29 

42 

43 

71 

C, 61 

26 

45 

65 

79 

91 

C, 47 

8 

19 

42 

78 

81 

D,58 

11 

14 

31 

54 

75 

A, 16 

23 

37 

49 

70 

90 

C. 46 

21 

25 

29 

55 

62 

B. 50 

26 

42 

71 

87 

87 

D,73 

10 

24 

41 

57 

84 

D, 34 

16 

29 

42 

43 

71 

C, 61 

41 

45 

74 

79 

97 

B, 58 

(Item 

unsatufactoryt noi 

; scored) 


D,65 

19 

73 

88 

90 

94 

C, 12 

20 

56 

92 

97 

98 

A. 7 

22 

45 

88 

94 

96 

A, 53 

28 

41 

78 

75 

85 

A. 73 

17 

20 

38 

52 

54 

B, 18 

8 

52 

79 

89 

94 

A, 25 

15 

49 

82 

92 

97 

A, 21 

17 

17 

35 

50 

53 

B. 43 

(Not included 

in test) 



(Used 

as example and not scored) 


C, 3 

15 

30 

75 

81 

92 

A, 85 

25 

38 

71 

82 

88 

B, 80 

19 

51 

82 

96 

96 

C, 57 

38 

38 

47 

56 

68 

A. 32 

6 

19 

26 

41 

55 

D, 35 

(Item 

unsatisfactory, not 

: scored) 


A, 13 

26 

35 

59 

79 

84 

A, 15 


tU 


*Tb mibel if wed to indicste thetf twrdt th« Mthor ifooid aet eoer iod e do ie • lift «f 
bet epea «hicb twt dan an avaibUo. 


[ 211 ] 



TBm AMO MEABUIIBMENTS 


APPENDIX D: Pereraate of CbSJreii is Each Grade Reeogniatiif Each of the 

Words T es te d - Contintui 


Words being Defined 



Grades 



Form 
and No. 
of Item 

4 

6 

8 

10 

12* 

^abolitionist 

13 

18 

77 

83 

92 

A, 48 

•anti-slavery 

26 

43 

84 

90 

91 

A. 24 

democrat 

33 

51 

87 

91 

92 

D. 69 

•federalist 

23 

40 

44 

48 

67 

C. 29 

political party 

50 

59 

92 

88 

95 

A. 80 

progressive 

IS 

23 

16 

23 

41 

D. 72 

republican 

33 

51 

87 

91 

92 

D,69 

socialist 

11 

25 

42 

50 

75 

A. 84 

•whig 

11 

21 

35 

SO 

65 

C, 42 

majority 

. 34 

52 

92 

94 

98 

B, 57 

minority 

unammous 

(Not included 
. 75 82 

in test) 

85 

90 

90 

B. 2 

ballot 

27 

43 

89 

92 

97 

A. 68 

election 

37 

70 

86 

89 

92 

B. 61 

polls 

34 

54 

87 

93 

93 

B. 31 

primary 

22 

41 

59 

77 

93 

A, 14 

suffrage 

13 

22 

58 

66 

92 

A. 18 

vote 

51 

62 

70 

70 

83 

A, 79 

ooBservative 

9 

20 

31 

52 

78 

D, 41 

•partisan 

19 

37 

52 

57 

66 

C, 62 

radical 

9 

20 

31 

52 

78 

D. 41 

tobbying 

2 

9 

24 

42 

63 

A, 47 

patronage 

8 

22 

26 

61 

67 

C, 60 

spoils system 

8 

22 

26 

61 

67 

C, 60 

bunness 

17 

43 

53 

65 

71 

C. 75 

commerce 

18 

52 

84 

95 

99 

B. 47 

commodity 

7 

18 

18 

38 

79 

C, 71 

company 

(Used 

as example and not 

scored) 


C, 2 

competition 

17 

30 

48 

73 

91 

C, 66 

coneumer 

14 

36 

72 

87 

89 

D,49 

exploit 

12 

15 

19 

19 

20 

A, 70 

expoR 

41 

75 

93 

94 

98 

C, 67 

fsetoty 

poods 

(Not included 
46 83 

in test) 

97 

99 

99 

A, 39 

Import 

inousny 

15 

66 

95 

97 

98 

A, 45 

14 

25 

85 

89 

95 

A. 40 

maBofitetute 


80 

97 

97 

98 

A. 38 

merdiandite. 


83 

97 

99 

99 

K 39 


18 

34 

S3 

77 

79 

D,70 


^Tlils spsHll is mmi «» wdiesM Asst word* th* satiwr venU not uam mdait w ■ list of 
till terms, bet (Wen nWeb test dm eie sTiiliUs. 
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APPENDIX D 


APPENDIX D: Percenttge of Children in Each Grade Recocntxiai EacJi of Ae 

Wordi Teeted— Con<i'a«ud 


Words being Defined 



Grades 



Form 

-.-.J XI-. 

4 



6 

8 

10 

12 

mo rio« 
of Item 

property 

(Not included in test) 




raw material 

rebate 

.. 40 
. . 17 

42 

30 

72 

35 

80 

67 

90 

86 

A, 43 
C, 64 

shipping 

(Not included in test) 


trade 

.. 18 

52 

84 

95 

99 

B. 47 

corporation 

8 

13 

18 

34 

S3 

B, 22 

monopoly 

. . 16 

29 

43 

65 

82 

C, 68 

trust 

8 

13 

18 

34 

S3 

B, 22 

employee 

.. 32 

48 

78 

92 

99 

C, 44 

employer 

(Not included 

in test) 




labor 

.. 34 

77 

87 

95 

99 

B, 10 

strike 

. . 30 

72 

91 

97 

97 

D, 25 

union 

. . 32 

48 

78 

92 

99 

C, 44 

bank 

(Not included in test) 




bankrupt 

. 61 

82 

96 

98 

98 

D, 23 

bond 

. 28 

74 

77 

90 

90 

C, 31 

capital 

S 

6 

44 

76 

89 

D, 29 

credit 

. 14 

56 

72 

91 

93 

D, 21 

crisis 

. 24 

33 

59 

80 

88 

C, 51 

depreciation 

. 28 

46 

74 

82 

88 

A, 41 

finance 

. 41 

73 

85 

90 

95 

B, 5 

inflation 

. 33 

34 

38 

38 

26 

B; S3 

investment 

. 35 

64 

83 

92 

94 

C, 72 

market 

. 48 

72 

65 

93 

97 

C. 65 

panic 

19 

26 

39 

49 

77 

A, 66 

speculation 

. 19 

26 

39 

49 

77 

A, 66 

stocks 

. 54 

81 

95 

96 

97 

B, 49 

communication 

24 

57 

93 

96 

97 

D. 22 

public utilities 

. 18 

34 

45 

62 

73 

D, 28 

transportation 

. 41 

63 

92 

92 

93 

A, 67 

prosperity 

. 28 

58 

80 

93 

93 

D, 47 

wealth 

. 49 

84 

84 

85 

86 

B, 4 

aristocrat 

. IS 

58 

83 

90 

96 

D, 15 

peasant 

. 32 

73 

91 

93 

97 

B, 72 

slave 

. 56 

67 

95 

95 

97 

A. 64 

society 

(Not included in test) 



communis 

homestead 

..45 68 94 98 

(Not counted because of error in 

98 

score sheet) 

C, 45 

pioneer 

.. 24 

69 

84 

89 

93 

C. 43 


*Tldi (ymbol if lued to indicate tliow eiardi the aBthor woold not now inclade in a Hit of 
ml tenof, Imm apon wUch teat data are availaUa. 
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TBSTS AND MEAStillEMENTS 


APPEI9DIX D: Perceatase of CtuMiea io Each Gnde Reeogtitzii^ Eadi of the 

Wnda Teetad — ComimeJ 


Wocdt being Defined 



Gradea 



Form 
and No. 
of Item 

4 

6 

8 

10 

12 

pUnution 

(Uaed 

aa example and not acored) 


C, 4 

•etdement 

50 

61 

90 

90 

96 

A. 69 

rural 

17 

29 

56 

73 

63 

B, 70 

urban 

17 

29 

56 

73 

63 

B, 70 

etnaui 

41 

63 

89 

95 

96 

D, 16 

inhabiunt 

27 

52 

83 

91 

97 

B, 17 

population 

27 

68 

94 

95 

98 

B, 67 

MlfOa a a a 

28 

71 

84 

85 

88 

D, 75 

race 

28 

71 

84 

85 

88 

D, 75 

mob. 

66 

75 

92 

92 

97 

B, 74 

riot 

39 

70 

86 

89 

89 

D, 64 

emigration 

11 

65 

86 

93 

97 

C, 36 

expanrion 

30 

30 

63 

67 

83 

D. 60 

uninigration 

33 

48 

89 

96 

97 

A, 51 

migration 

29 

42 

72 

87 

96 

D, 17 

emanopatton 

22 

25 

59 

69 

86 

B, 24 

fraedom 

84 

89 

99 

99 

99 

A, 33 

independence 

37 

42 

76 

76 

85 

A, 63 

Ubeiv 

84 

89 

99 

99 

99 

A, 33 

oppretaioa 

5 

15 

25 

40 

59 

D, 67 

education 

methution 

invention 

(Not included in teat) 
(Not indudedin teat) 

. 39 52 89 

90 

96 

A, 65 

reform T 

. 19 

44 

53 

65 

85 

D» 20 

people 

private 

puWc 

public opinion 

atandarn of living 

(Not induded 
(Not induded 
(Not induded 
, 16 47 

in teat) 
in mat) 
in teat) 

81 

95 

96 

B. 48 

. 23 

70 

87 

96 

96 

B. 33 

Wtraty 

31 

36 

52 

55 

58 

D. 42 

iBetil 

32 

33 

66 

80 

96 

B. 37 

«iiivaHd 

6 

II 

13 

24 

24 

B, 32 

ioitieu 

39 

52 

80 

87 

92 

A, 61 

Lgd 

ri^ta. . 

(Not induded in teat) 

25 54 76 

85 

88 

B,25 

uiKoiiatitut^d 

6 

11 

13 

24 

24 

B. 32 


*Tldi tyaM It to indktt* Aon ^ MdMr «Mdl Mt BOW iada4t to a Bk if 

dal (anH, lot atm olldi taaa dan an avaflaUa. 
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APPENDIX D 


APPENDIX D: Percentage of Chfldren in Each Grade Recogniaing Each of the 

Wordi Tcfted— Cofirieiuirf 


Words being Dehned 



Grades 



Form 
and No. 
of Item 

4 

6 

8 

10 

12 

alien 

io 

39 

88 

97 

99 

A. 11 

ddzen 

48 

63 

74 

86 

76 

B. 39 

exile 

H 

33 

63 

86 

95 

B. 36 

nationality 

48 

67 

90 

93 

96 

C, 25 

native 

43 

65 

81 

90 

96 

D, 10 

naturalization 

37 

60 

81 

93 

% 

D» 26 

appeal 

(Not included in test) 




case 

(Not included in test) 




convict 

(Not included 

in test) 




crime 

. 37 

75 

88 

88 

90 

D, 77 

decision 

17 

34 

83 

93 

97 

A, 57 

execution 

18 

39 

84 

89 

92 

A, 72 

injunction 

7 

19 

42 

55 

69 

B, 68 

judge 

(Used 

ai an example and 

not scored) 

A, 5 

jury 

29 

74 

84 

89 

95 

B, 64 

testimony 

28 

61 

70 

85 

92 

C, 73 

verdict 

17 

34 

83 

93 

97 

A, 57 

violation 

21 

53 

93 

94 

94 

D,44 

witness 

28 

61 

70 

85 

92 

C, 73 

court 

64 

88 

95 

97 

98 

B, 52 

jurisdiction 

21 

26 

48 

59 

85 

A, 74 

supreme court 

36 

48 

78 

88 

97 

B. 38 

bribery 

20 

34 

70 

88 

96 

A, 52 

corruption 

16 

27 

32 

57 

73 

B. 34 

fraud 

31 

56 

63 

89 

94 

D. 8 

graft 

16 

27 

32 

57 

73 

B, 34 

conspiracy 

2 

6 

19 

34 

39 

B, 75 

insurrection 

5 

13 

41 

56 

77 

B. 71 

rebellion 

(Used 

as example and not scored) 


C. 5 

revolt 

5 

13 

41 

56 

77 

B, 71 

revolution 

(Used 

as example and not scored) 


C. 5 

secession 

16 

21 

56 

58 

89 

B. 13 

sedition 

2 

6 

19 

34 

39 

B. 75 

■mug^e. . , 

37 

52 

80 

87 

92 

A. 60 

treason 

30 

48 

76 

93 

94 

C, 34 

allies. 

34 

37 

73 

82 

91 

B. 73 

^belligerents 

9 

34 

36 

49 

56 

B. 23 

enemy 

14 

38 

64 

76 

92 

B. 12 

hostile 

14 

38 

64 

76 

92 

B, 12 


*TIim ^niM if tued to indicate tboee wordt die uitkor would not now include in a Bat of 
dal tenu, but upon wUch taet data ate availabla. 
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TESTS AND MEAStMLEMENTS 


APPENDIX D: Percentaie of Quldicn in Etch Gride Recogniang Eidi of the 

Wordi Teeted— Conrratwd 


Grade* Form 

Word* being Defined and No. 

4 6 8 10 12 of Item 


•pirate SO 7l 9S 96 98 A, SS 

army ^ot induded in test) 

commander (Not included in test) 

confederate 14 37 67 82 88 B, 4S 

cruiser 31 42 78 80 88 A. 78 

S etwrat (Not included in test) 

eet 48 82 91 91 94 C, 79 

force* (Not included in test) 

naval 41 82 96 96 99 B, 60 

navy 31 42 78 80 88 A, 78 

officer P^ot included in test) 

reenforcements (Not included in test) 

service (Not included in test) 

troops (Not included in test) 

submarine 76 97 97 99 99 D, 6 

marine IS 39 52 58 72 D, 7 

mUitia 2 12 22 38 44 A, 83 

Foeruit 20 20 47 66 82 A, 35 

iddier (Not included in test) 

veteran 37 84 93 98 99 C, 21 

volunteer 41 73 91 93 97 B, 63 

draft 38 62 81 87 95 B, 3 

mobiliaation 10 22 29 43 70 C, 55 

•aggression 5 17 45 51 79 D, 9 

attack P^ot used in test) 

battle (Not included in test) 

blockade 43 43 69 72 83 B, 79 

•bombardment..' 33 53 66 77 88 A, 56 

campaign 19 30 50 59 81 D, 19 

contraband 31 52 57 58 68 D, 50 

defensive 41 80 90 94 97 C, 49 

embargo 9 IS 63 58 ^ A, 81 

Evacuation 12 16 24 37 39 B, 9 

•fortification 33 53 66 77 88 A, 56 

invasion 5 17 45 51 79 D, 9 

*mfsacfe 39 75 89 95 98 D, 46 

miUtary 34 69 81 86 95 Q 54 

•fttunitiont.ir 18 34 56 59 77 D, 33 

•occupation, 12 16 24 37 39 B, 9 

offensive 41 80 90 94 97 C, 49 

tiene 6 23 59 73 81 A, 54 


■TUmnabol is wMi, to indicue tfapw word* dbe muhor would mat mom iocliide iw a lilt of 
tiai tsnasy Imk t^iw wlucli tatt dan an SvaiiaUs. 
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APPENDIX D 


APPENDIX D: Percenuge of Children in Each Gnde Recognising Etch of the 

Words Tested — CoHttnutd 


Words being Defined 



Grades 



Form 
and No. 
of Item 

4 

6 

8 

10 

12 

'strategic 

13 

22 

S3 

61 

77 

D, 57 

surrender 

37 

86 

91 

94 

96 

D. 14 

victory 

(Not included in test) 




war 

(Not included in test) 




armistice 

24 

24 

26 

29 

35 

A. 62 

disarmament 

33 

60 

79 

87 

88 

D, 56 

indemnity 

18 

28 

46 

65 

82 

D,32 

reparations 

23 

23 

65 

74 

91 

B, 59 

boundary 

43 

89 

95 

% 

97 

B. 77 

continent 

33 

79 

91 

95 

97 

C. 24 

'continental 

4 

10 

11 

13 

20 

D» 80 

coast 

(Used 

as example and not scored) 

A, 2 

frontier 

(Item unsatisfactory) 



B. 66 

prairie 

20 

54 

75 

86 

89 

B, 54 

district 

(Not included in test) 




region 

(Not included in test) 




section 

(Not included in test) 




discover 

S3 

71 

90 

93 

97 

C. 22 

expedition 

S3 

71 

90 

93 

97 

C, 22 

exploration 

(Not included in test) 




navigation 

41 

71 

86 

94 

97 

B, 6 

voyage 

59 

97 

98 

99 

99 

D,5 

conservation 

19 

30 

46 

58 

68 

C, 23 

natural resources 

9 

37 

86 

91 

95 

A, 19 

agriculture 

, 19 

92 

97 

99 

100 

C, 7 

irrigation 

. 37 

85 

93 

96 

96 

C, 6 

reclamation 

9 

33 

61 

79 

84 

B, 7 

•Pan-American 

24 

29 

32 

46 

64 

C, 74 

clergy 

22 

27 

50 

76 

86 

B, 27 

missronary 

33 

80 

93 

93 

97 

C, 70 

creed 

12 

23 

68 

76 

90 

A, 82 

•denomination 

. 28 

42 

SO 

55 

74 

Q 17 

•heresy.. 

11 

24 

50 

62 

72 

D, 68 

intolerance 

(Not included in test) 




persecution 

, 11 

24 

so 

62 

72 

D. 68 

tolerance 

, 26 

27 

41 

58 

76 

A. 46 


"This qrmbal k lued to indicate tboec word* the anthoT would aet wow mdude in a Uet of 
urm$, liarvpMi wUch ten data are aTaiUUa. 
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APPENDIX D: PwcenUte of Children in Each Grade Recoenning Each of the 

Words Teated— Crarimird 


Grade* Form 

Word* beini Defined • and No. 

4 6 8 10 12 of Item 


cadiolidtm 26 26 58 63 67 A, 27 

piotMtantinn 26 26 58 63 67 A, 27 

eninde 11 40 74 86 92 C, 10 

papacy 23 30 52 79 88 B, 62 

pope 23 30 52 79 88 B, 62 

•ndent 11 61 70 76 87 C, 8 

century 25 M 92 94 98 B, 14 

current 28 57 84 86 91 C, 14 

decade 18 26 46 49 69 C, 13 

era 25 41 44 72 83 C, 50 

event 21 38 64 68 84 A. 76 

mediaeval 19 32 47 72 81 D, 37 

modern 11 61 70 76 87 C, 8 

p^ 25 41 44 72 83 C, 50 

hUtorv (Not included in test) 

Fsoorda (Not included in test) 

propa|anda 11 25 48 71 80 Ci 33 

publicity (Not included in test) 

civilised 28 40 43 62 84 C, 63 

prioutive 28 40 43 62 84 63 

movement (Not included in test) 

pcecedent 12 13 23 30 46 A, 22 

tradition 12 13 23 30 46 A, 22 


CLASSIFICATION OF GRAPHS 


Number of 
Each Type 


1. Curve* rising from low achievement in Grade IV to high achieve- 

ment in Grade XII 70 

2. Curves ihowing slow start, but rapid subsequent rise 31 

3. Curves showing rapid initial rise, and then slow improvement 32 

Curves showing high mastery in all grades; little increase 45 

51 Curves dhowin^ very dow, but regular, progress 36 

^ Curves oimttimng a marked plateau 19 

T. Curve* nowing a high start, but iitde subsequent improvement. . . 6 

8^ Curves Arming marked gains between Grades VI and VIII, but 

Btde nin daewhere 80 

9, Curves nowim almost no^ gains at any levd 14 

l(k Curve* jp whidi a higher grade scores bdow the previous one n 
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THE WESLEY TESTS IN SOCTAL TERMS 

EdCAJL B. WEftLET 
University of Minnesota 

I. The Problem 

The efficacy of obje^ive tests in the measurement of informa- 
tion has been widely recognized. Their success in measuring un- 
derstanding has been by no means so complete. Specific informa- 
tion which clusters around individuals, events, circumstances* and 
symbols has been measured quite successfully by the new-Cype 
tests, but even high scores do not prove that the testee understandj 
the facts which he knows. Can objective tests be so constructed as 
to measure, or at least indicate, that the testee both knows and 
understands? The attempt to answer this question in the affirma- 
tive led to the construction of the Wesley Tests in Social Terms.* 

At the outset it was recognized that the usual array of names, 
dates, and events would test little beyond information. Sp>ecific 
facts, having no wide application, are likely to be practically inert, 
whereas terms expressive of sodal relationships have wide appli- 
cation and so are relatively functional. Consequently the first task 
was to select those terms which are functional, and the second task 
was to construct tests which would demonstrate whether the testee 
knew the term as a mere verbalism or whether he understood its 
true significance. 

II. Selection of Terms 

The word list from which the test terms were taken was com- 
piled under the supervision of Professors Krey and Kelley in the 
fall of 1929. An effort was made to select those words and terms 
which have wide applicability. This involved the elimination of 
proper names and highly technical words and phrases. The bases 
of selection led naturally to the selection of words which function 
in more than one situation. Consequently the list was not limited 
to history or to any one of the social studies but was highly com- 
posite in its nature. When completed it contained about 4,000 
words and terms. 

^Wedey Ttst in Pditual Terms, Forms A, B, C, and D; Lesley Test in Soeini 
Terms, Forms A and B; Wesley CaUef/t Test in Social Terms, Forms A and B, 
New York, Scribners, 1932, 1933. 
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After it was completed it was sent to thirty-five college instruc- 
tors, twenty-seven of whom cooperated in the evaluation of the 
words and terms. They were asked to indicate if the entering 
freshman should have (1) a fairly complete understanding of the 
word or term, (2) a partial knowledge, or (3) if no knowledge of 
the word or term was expected. The twenty-seven Judgments on 
each term were combined into a single average rating which indi- 
cated the supposed utility and, by implication, something of the 
difficulty of each term. Thus the list of terms in the social studies 
represented the combined judgment of more than forty persons, 
including the working staff. 

III. Wesley Test in Political Terms 
The compilation of a word list and the evaluation of its contents 
in terms of utility and difficulty greatly facilitated the selection of 
items for the test. In addition to the official word list, considerable 
help was obtained from others which had already appeared in 
print.* For the purposes of the fim test the author took the official 
(American Historical Association)* list and selected from it the po- 
litical terms, taking care to omit (1) very simple words, (2) obso- 
lete terms, (3) highly technical terms, (4) exceptionally difficult 
terms, and (5) phrases whose members could be utilized separately. 
Political was interpreted to include military, diplomatic, legal, and 
other terms closely related to government. To these words, num- 
bering 1,190 were added 84 terms drawn from (1) Stephenson’s 
list, (2) from a^list of political terms on the junior high school level 
(prepared by Miss Edith Ware from the original A.H.A. list), and 
(3) words which suggested themselves to some member of the Com- 
mission staff as possessing great utility. 

Words of varying degrees of difficulty were selected from this 
list of political terms to be utilized in the first edition of the Wesley 
Tests. Some terms were omitted because of the difficulty of reduc- 
ing them to objective form. Special effort was made to utilize the 

*'nte two wbidi were espedally useful were Orlando Stephenson, ‘The Special 
VocabuUlT Gvics," Journal ^ Educational Rtsearek^ XVIII (Nov., 1928), 
297''"IIM:k u| EL E. Eubanks, "The Concepts of Socidosy," Social Foreej, V 
391-394. 

*SiiLIIP9endix to volume. 
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terms which appeared in all the word lists. A larger proportion of 
the terms in the important and difficult group was utilized than in 
the middle groups. Terms from local, state, national, and inter- 
national fields were used ^ insure a representative sampling. 

Edition I was prepared in the summer of 1930. It consisted of 
274 items in a variety of forms: (1) historical illustration, (2) iden- 
tification from a rather full description, (3) matching (three types 
within this general group), (4) substitution (both of the recall and 
control type), (5) completion (both of the recall and control type), 
and (6) best-answer. In September, 1930, Edition I was given to 
three hundred and ninety-seven freshmen at the University of 
Minnesota and to about two hundred seniors in high school. Under 
the direction of Professor Donald G. Paterson, a statistical study 
was made of the college papers and on the basis of these results 
items were selected for Edition II. 

The results of Edition I were studied by means of both statistics 
and logical analysis. The high school papers as well as the college 
papers furnished material on which to base tentative conclusions. 
The process of evaluating the various forms was a prolonged task 
involving both statistical results and judgment. The true-false 
type was not used, for in the field of the social studies most truths 
are relative, and it is difRcult to phrase an item involving either 
clear and unquestioned truth or palpable error. The form which 
called for an historical illustration proved to be a good measure of 
both information and understanding, but the form required the 
careful attention of one familiar with the field and, in the final 
analysis, rested upon judgment. In other words, it was not suscep- 
tible of mechanical objectivity. The identification from rather full 
descriptions required enormous space and was likewise devoid of 
complete objectivity. The matching forms seemed to allow too 
large an element of guessing, especially after the first few opera- 
tions had been performed. The same objections may be made to 
the substitution types. The completion type works well when con- 
cerned with specific facts, but is much more difficult to construct 
when dealing with functional ideas and is lacking in objectivity in 
its answers. On the other hand, the best-answer type with five op- 
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tiont was sdected as the form which caHed for the highest degree 
of discriminatioti. The testee, being aware of the fact that the cor- 
rect answer is before him, needs to employ rote memory scarcely 
at all. Many items contain acceptable options but only one which 
is clearly best. The form allows the use of false, but attractive, 
associations, of partially correct ideas, and of a variety of options 
based upon similarities of sound and form. The student who picks 
his way through such an array of misleading suggestions and in- 
sists upon choosing the best answer has, allowing for the small per- 
centage of chance, demonstrated his understanding more convinc- 
ingly perhaps than by any other type of objective form. The di- 
rector of the Commission and most of the staff agreed that the 
best-answer type showed most promise, and it was accordingly ac- 
cepted as the one to be used most extensively. 

The items which survived Edition I were utilized in Edition II. 
No new items were introduced, but some were changed from other 
forms to the best-answer type. Two parallel forms of fifty items 
each were prepared and given to 1,132 high school and college 
students. Edition II was given to the following numbers in the 
various grades: 


Grade 

Numbers 

11 

252 

12 

376 

13 

409 

.14 

26 

'IS 

39 

16 

30 


The returns from Edition II were studied under the direction of 
Professtik Truman L. Kdley. Forty of the hundred items seemed 
to have ^muine promise. They were selected on the basis of the 
stati8tici| tmdts and were published in four forms of ten items 
ewril. Emi^ form has a rdiability coefficient of .68, which is suffi- 
li^ the measurement of groups, but not sufficiently 
^ imasurement of individuals. When all four forms are 
linedi^ov|ever« the resulttfig score is sufficiently reliable for individ- 
Btl dia^p^sis. The test is standardized for the 12th grade, but may 
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be used with the three grades before and after. Its usefulness ex- 
tends from the 9th grade to the end of the sophomore year in 
college. 

The validity of a test is the degree of success with which it mea- 
sures what it purports to measure. Validity, in the final analysis, 
must rest upon judgment, although statistical returns will either 
confirm or tend to invalidate judgment. The validity of the Wtsley 
Test in Political Terms rests upon the following: (1) The material 
utilized in the tests was taken from official and approved sources. 
(2) The material unquestionably belongs within the held of govern- 
ment. (3) Only those items which differentiated rcliahly between 
the superior and inferior groups were retained in the final forms. 
(4) The correlations between the Wesley Test in Political Terms and 
other tests (reported fully in the Examiner’s Manual accompany- 
ing the test) show that it measures to some extent what intelligence 
tests measure, and to a somewhat greater degree it measures what 
reading and civic tests measure. But the correlations, even when 
corrected for attenuation, fall far short of perfect relationships. 
The conclusion is that the Wesley Test measures to a considerable 
degree what other tests do not measure. The assumption is that it 
measures understanding in the social studies. 

IV. Wesley Test in Social Terms 

The same lists used in the construction of the test in Political 
Terms were used in constructing the test in Social Terms. The 
lists by Eubanks and Stephenson and indexes of texts in sociolc^, 
economics, and political science were consulted in the determina- 
tion of doubtful cases. This test differs fundamentally^ from the 
first in the respect that it utilizes terms from all the social studies 
instead of being confined to political history and government. 

Several of the items used in the trial forms of the,^tsst in Political 
Terms were utilized in what might be called Edition of the 

concluskms as to form and technique were so utili|IM<, 

The resulting study showed that 160 items pdacMO® lllljfli* 
factory power to discriminate. They were accordin^y aa:a^||(i3;& 
two parallel forms of 80 items each All that was said jdHMK 
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validity of the test in Political TermSy save the correlation with 
Other tests, holds true in regard to the test in Social Terms. 

V. Wesley College Test in Social Terms 


In addition to the A.H.A. list and college texts, another source 
of first importance was used in the construction of the College Test. 
In the summer of 1932 the author had charge of students who en- 
rolled for a symposium on the Social Studies which was held at 
the College of Education at the University of Minnesota. The lec- 
tures were combined into a series to constitute a course for which 


college and graduate credit was given. Two rather elaborate tests 
were constructed in connection with the course. A careful evalua- 
tion of the items was made by Mr. Wilbur Murra of the University 
High School. Thus, some sixty items of recognized validity were 
available as a nucleus for the college test. The college test utilizes 
terms from all the social studies. It is, in fact, a more advanced 
form of the test of political terms which was first evolved for the 
twelfth grade. The college test is standardized for the end of the 
•ophomore year in college, but may be used for all years from the 
12th grade to the senior year in college. 

Another factor of undoubted significance was the prolonged, 
critical, and patient help of members of the social science depart- 
ments of the University of Minnesota. They added materially to 
the validity of the test by their careful evaluation of each item. 

The preliminary forms, consisting of 137 items in each of two 
forms, vrere given to students in four widely scattered colleges as 


follows: 


Class 


Numbers 


Sophomores 44 

Juniors 101 

Seniors 87 


Total ?32 

A ttatisttod study of the results was carried on undei the direction 
of Eh*. F. H. Finch, Head^of the Personnel Department of the Uni- 
vcxiity ofldinnesota High School. From this study, 210 items of 
power to discriminate were arranged in two parallel 
ll5 items each. 
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What was said as to the validity of the test in PoliHeal TVrmr, 
save the correlation with other tests, applies equally well to the 
college test. 

‘VI. Conclusions 

The varying connotations of words is a familiar experience to 
adults. Even the simplest words enshroud themselves in habili* 
ments which strike persons other than the user as strange or even 
outlandish. When one has passed beyond the simplest and most 
elementary uses of a word, it begins to assume these varying con- 
notations. The most careful student of language is likely to be 
most keenly aware of the various meanings and complications of a 
word or term. The person who has grasped the word as little more 
than a verbalism is likely to be the one who thinks he knows its 
full and correct meaning. As one advances in experience and learn- 
ing, he realizes the endless stages or levels of meaning and so be- 
comes less dogmatic as to final connotations. 

New words, upon their initial appearance, seem to be more or 
less mechanical or arbitrary names, which are for the sole purpose 
of identifying the thing or idea to which they are applied. In this 
initial stage they serve the purpose in a manner as devoid of per- 
sonalities as if they were numbers. A toy horse might just as well 
be called by another name so far as the child is immediately con- 
cerned. Constitution is an arbitrary word for the pupils of the mid- 
dle grades until it becomes more than a mere identification tag. 
Imperialism is a highly abstract idea, perhaps only a sound or a 
series of letters, to the high school boy who encounters it for the 
first time. In each instance the word may remain as a mere name 
or it may become the symbol of a rich and many-colored idea. 

But even verbalisms have their utility and constitute the first 
step in the acquisition of an understanding of a word. As the pupil 
moves through the successive grades he acquires both fuller «m- 
notations of familiar words and introductions to new ones; so in 
high school and college, as well as in the elementary grades, the 
teacher is confronted with the problem of the learning process 
through all of its stages. It is probable that the high school stu* 
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dent hag (Mgun to gn$p the higher JeveU of some words and so is 
prepared to recognize the poMibilitieB of enriched connotations. 
These advances* however, are by no means regular, nor do they 
necessarily correlate highly with cither grade classification or in> 
telligence. The measurement of these levels of understanding at 
opposed to mere verbalisms it a task that requires the constant 
attention of the teacher of the social studies.* 

An ideal test would measure not only the highest but the inter- 
mediate levels of understanding. Each item would be so graduated 
as to reveal the level which the testee has reached. It would also 
take account of verbalisms, since they probably indicate the first 
step on the road toward a full understanding. Many tests do pro- 
vide for this measurement of verbalisms or the recall of rote infor- 
mation: fewer of them call for the discrimination which is involved 
in an understanding of ideas; and no test in the social studies pro- 
vides for graduated scoring within the various options of the items. 
The fFrslry Tests do not systematically allow for this graduated 
degree of correctness within each item; in this respect they would 
seem to fall short of what is possible in dealing with some terms. 
The extent to which this graduated scheme can be applied is yet 
undetermined. 

In spite of the limitless nature of words, they do have, at vari- 
ous levels, fairly definite meanings, the understanding of which can 
be tested. The Wesley Tests attempt to measure this understand- 
ing. They are.designed to indicate the degree of comprehension or 
level of understanding which a student has of a particular word. 
The simplest, most obvious meaning is not always that which is 
sought; frequently a meaning further toward the periphery is the 
one involved. The care with which the tests were made, the rigid 
ctidcism dirough which they passed, and the thorough revisions 
which were made, together with the preliminary reports upon their 
use, seem to indicate that they have to some considerable extent 
achieved l|ie purposes fpr which they were designed. 

tA RdtSf thtcusiton of a parallel idea may be found in Ednr 6. WeiW, 
^seti ex Ifeta in die Social Studies," Historical Outlook, XXIV, 28-30; A. C. 
Ibqr and Einr B. Wedey, "Dom the New-Type Test Measure Resulu of Instruc- 
^ Studies," Ibid., XXIII, 7-21. 
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THE KELTY-MOORE TEST OF 
CONCEPTS IN THE SOCIAL STUDIES 

A 

Mary G. Kelty and Nblle E. Moore 
I. Inadequacy of Children’s Concepts 

Language and thought have long been recognized as interde- 
pendent. The purpose of language is to express thought; and 
thought, on any except the simplest plane, is impossible without 
language. “The relation between thought and language becomes 
so intimate that the two are psychologically inseparable.” 

Yet it is fallacious to assume that children understand an idea 
because they can repeat the words in which the idea is embodied. 
Those words may not, in any way, link themselves up with the 
children’s background of experience, and may therefore completely 
lack meaning, even for those who use them glibly. 

Many investigations have concerned themselves with children’s 
concepts. G. Stanley Hall in his study, “The Contents of Chil- 
dren’s Minds, in 1890 showed that children habitually used 
words which had little or no meaning for them. Earl Barnes* in- 
vestigated specific terms in history and found that children under- 
stood poorly all the terms except those which touched their own 
experience directly. Chambers* went a step farther and classified 
the causes of children’s misapprehensions of terms, such causes, for 
example, as similarities in sound or form. Scott and Myers* sug- 
gested, as a result of their investigation, that children’s errors ex- 
pressed language difficulties in which “the whole modus operandi of 
thought is involved” and that “much of school learning is only 

*G. Stanley Hall, Sonu Aspects of Child Life and Education, Ginn k Co., 
1907 Ed., 1-52. . . « . . 

*£ari Bamct, Studies in Education, Vol. II. Stanford Univeriity, California, 
1897. p. 60. 

•W3I G. Chambers, “How Words Get Meaning," Pedagogical Seminary, XI 

(19(Hr30-50. „ ^ « 

♦Flora Scott and Garry C. Myers, “Children’s Empty and Erroneous Concep- 
tioat of the Commonplace," Journal of EductUional JUsearch, VIII (1923) 327-34. 
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word manipulation/* Melezer* tested the concepts of children in 
succeeding grades, tanning with the fifth, to determine both their 
specificity and their completeness. He found that growth was 
steady throughout the grades, but that misapprehension and 
vagueness were common. Ayer* showed that the progress of many 
children in studying history was blocked by their lack of compre- 
hension of abstract thoughts and words, as well as by "literary 
embellishments" and involved sentences. Piaget* pointed out in his 
reports another factor of significance, t. r., that nine- to eleven- 
year-old children not only did not understand many words and 
sentences which they used fluently, but that they believed they 
did understand the ideas and asked for few explanations. 

After considering such investigations, one is inclined to agree 
with John Dewey that vague meanings are “the source from which 
flow roost bad intellectual consequences." To determine whether 
or not children possess clear-cut and vivid concepts of the mean- 
ii^ connected with words becomes, therefore, a problem of vital 
omcem to the teacher. Measuring or testing concepts is a means 
of measuring or testing thought. 

II. Lists of the Most Important Social Concepts 

In order to be able to test concepts, the teacher must under- 
stand the general processes by which concepts are developed. A 
discussion of these processes, however, lies outside the field of con- 
cern of the present article. 

If, in addition to understanding the development of concepts, 
the teachw is to be able to ascertain whether or not children have 
a real understanding of terms essential to the social studies, she 
should possess a guidance list of the most important social con- 
cq>ts. Tlw assembling of such a list ought not to be left to the 
individual judgment of each teacher. It ought at least to represent 
a podling of the judgments of experts. 

*Hwinaa Mdtaer, Ckildr^’t Social Cottcrpu. Bureau of Publications, Teach- 
ers C(dh^ CalanUna University, 1925. 

*Adwi|e M. Ayer, Some Dijkfulties in Elememary Seko<d History. Bureau of 
PddicatiQnii Tsadiers Cdl^et .Columbia Univertity, 1926. 

£*»t"«ge nnd Tkonght of the Child. Harcourt Brace and Co., 

1 ^); * ^ 
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Sudi a list has been prepared for publication as a part of the 
report of the Commission on the Social Studies of the American 
Historical Association. Numerous lists have also appeared for the 
separate subjects, and| there are even some lists for special subjects 
at a particular school level.^ 

For the purpose of the present investigation (the Kelty-Moore 
Test of Social Concepts), the above-mentioned list of the Com- 
mission on the Social Studies was used as a basis. To this list were 
added at the lower level many simpler terms and concepts, and 
many terms from Industrial History, from the Kelty list referred 
to in footnote 1, p. 233. 

III. The Kelty-Moore Test; Its Construction and Validity 

In order to test the understanding of concepts on the part of 
children in the middle grades, as well as in the higher grades, the 
simplest possible form was chosen.* (See pages 9 if.) The procedure 
was as follows: 

(1) The composite list of words issued by the Commission on 
the Social Studies (referred to in the second paragraph above) was 
first divided into groups of terms which, in the opinion of the 
testers, could be satisfactorily taught to children in the following 
grades: fourth, fifth, sixth, and junior high school. The simpler 
concepts and those used primarily in the early periods of American 
History were assigned to the fourth grade; those from later Amer- 
ican History, to the fifth grade. Since the most common subject 
of history study in the sixth grade is European Background, the 
terms used chiefly in that field were assigned to the sixth grade. 
More difficult terms in American History, including complex politi- 
cal and economic concepts, were assigned to the junior high sdiool. 

It is obvious that many fourth grade to junior high school pupils 

*The lull for history have b«en sununarized in TJu Journal of Edueational 
RtstarcK XXXIV (December, 1931), 335-49. For the A. H. A. list see appendw 
to this vdume. 

*Ratber than such forms as used by Garr and Gifford in the Second Yearbook 
of die National (^un^ for the Social studies (1932), p. 164; or hy ShaSin in the 
Journal of Education^ Psychology, XIX (1928), 41-44. 

[ 229 ] 



TESTS AKD HBASUILEMENTS 


do not know the specthc terms assigned to their grades because 
they have never been exposed to the materials. The testers, how> 
ever, are convinced from years of experience in teaching those par- 
ticular terms and from informal testing following the teaching, 
that average children of the grades mentioned can understand the 
terms if they are taught them directly. 

(2) Each of the graded lists of terms described above was then 
divided into groups according to meaning. The purpose of such 
grouping was to assist in random sampling and to insure the in- 
clusion of alt the different types of words. The scheme of classifica- 
tion according to meaning was: 

Objects with which children have had experience. 

Objects with which children have not had experience. 

Names of actions. 

Mental attitudes. 

Condition or state of being. 

political and economic concepts. 

Miscellaneous (words which did not belong in any of the other 
categories). 

*(3) Test-lists of fifty-six terms each were then selected for all 
the grades under consideration, by choosing eight terms from each 
of the seven categories. The total number of words in each cate- 
gory was divided by eight, and the count proceeded on the quo- 
tient thus secured. 

(4) The list* of fifty-six terms for each grade was divided into 
two approximately equivalent forms, X and Y. 

(5) With the assistance of the Standard Dictionary and Web- 
ster’s Dictionaiy a correct statement of the meaning of each term 
wit written. An effort was made to avoid the exact dictionary 
deSnttion. A second statement of meaning was also prepared, 
whidi was nearly right, or which, in the judgment of the testers, 
rei^esented errors that children would be likely to make. 

A l^t of the fifty^six terms for each grade (fourth to junior 
sdioal) was then given to one group of children from each 
with the request thlt they write a statement of what they 
. likraHEht etch term meant, from these responses, enough wrong 
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Items were selected to make a total of five items for each vootbu- 
lary-term. 

(7) Forms X and Y for each grade from the fourth to the junior 
high school were theif* mimeographed. Each test was adminis- 
tered to two groups of children in each grade (about fifty children 
in most cases). 

(8) The results were tabulated. The tabulations of certain of 
the items called the attention of the testers to misleading state- 
ments and to other unwise complications in the test. Accordingly, 
changes were made where necessary, to insure that the test should 
measure only vdiat it was designed to measure, and that the con- 
cept itself should present the only difficulty. 

(9) The completed test-forms were then presented to three lead- 
ing authorities on the teaching of the social studies, who criticized 
them on the basis (1) the importance of the terms, and (2) the 
adequacy of the statements from considerations of scholarship. 

(10) These tests were then administered to a random sample 
of 100 fourth grade pupils, a random sample of 1(X) sixth grade 
pupils, and a random sample of 100 eighth grade pupils. The 
items were split into halves as follows: (1) items 15-36, 46, 48-51, 
66-71, 79; (2) items 1-13, 37-42, 44, 53-54, 56^, 63, 72, 74-77, and 
correlated to determine their reliability. The poorer items which 
had been rejected during the process of construction were also 
split into halves and correlated as follows: (1) items 45, 47, 52, 
64-65, 80, and (2) items 14, 43, 55, 61-62, 73, 78. The correlations 
resulting are tabulated herewith. 


RELIABILITY OF ORIGINAL KELTY-MOORE TEST ITEMS 



Better 

Itetnt 

Equivalent 
Ruiability 
per Item 

Poorer 

Itemi 

Equivalent 

Reliabdtty 

100 ith tnde pupili 

.749 

.082 

.028 

.0014 

100 6th mde pupili 

.765 

.069 

.290 

.059 

100 8th grade pupili 

.790 

.101 

.329 

.070 


Seventy itmns were thereupon chosen. These were divided into 
two forms (A and B) of thiity-hve items each for individual jdafie- 
otmt, die reliabiUty per form being in the nei^bmfiood of .77. 
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An seventy items in one test furnish a reliability of .85 in the fourth 
grade, .87 in the sixth, and .90 in the eighth. 

In addition seven approximately equivalent forms (Forms 1-7) 
of ten items each were arranged for survey purposes. The reliabil- 
per grade group was about .45. 

(11) The eighty items from which were later chosen the seventy 
items which a>nstituted Forms A and B, and Forms 1, 2, 3, 4, 5, 
6, 7 were administered to pupils in grades four to nine in schools at 
Dearbome, Michigan; Montevallo, Alabama; Reading, Massachu- 
setts; Washington, D. C.; Wilmette, Illinois; and Winnetka, 
Illinois. The tentative norms upon Forms A and B, and Forms 
1, 2, 3, 4, 5, 6, 7 were computed from this tryout. They appear 
below. 

TENTATIVE NORMS ON THE KELTY-MOORE TEST OF CONCEPTS IN 

THE SOCIAL STUDIES 


Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 9 


Forin A Of B 

HEI 

13.0 

18.6 

23.4 

26.0 

26.2 

Fomii 1, 2t 3, 4, 5. 6. 7 

2.6 

3.7 

5.3 

6.7 

7.4 

7.S 

No. of Pu(hIi 

216 

199 

170 

260 

316 

181 


IV. The Uses or the Test 

Forms A and B and Forms 1-7 serve to give a general picture of 
children's understanding of essential terms in the social studies, 
chosen by random sampling. They assist teachers in determining 
how a given group compares, in general, with other groups through- 
out the country. They may point out the necessity of revising the 
local program of instruction, in such respects as wider use of ex- 
tensive reading, more inclusion of actual experiences, more direct 
instruccilDn in word-meanings, and more systematic testing of the 
undetatfnding of terms. 

If dbe problem of testing children's concepts is to be attacked 
eonstrudtively and continuously, however, the Kelty-Moore Test 
wiR be i|Brviceable only as a guide and a measuring stick. The real 
sduticnlof ^ problott' lies in the breakingnip of the complete list 
^ ioidtlstadses concepts into small groups, whose meanings can be 
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developed in connection with a small unit of experience.* This short 
list should be made the subject of direct instruction, such as the 
supplying of experiences or the recalling of experiences. After a 
proper interval, the short list should also be made the subject of a 
test similar to the Kelty-Moore Test.* Where necessary, the in- 
struction of individual pupils should be repeated. Only by such a 
systematic, continuous program can teachers be assured that chil- 
dren possess real concepts rather than a list of words empty of 
meaning. 

'For eumple, as in Mary G. Kelty, Teaching American History the Middle 
Grades of the Elementary School. Ginn and Co., 192^ pp. 56, 61, 64, 68, 72, 76, 80, etc. 

'May be secured through Charier Scribner's ^ns. 
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TESTING OF SKILLS 

'Die objective of skills was described by the planning committee 
as follows: 

**Skill in the use of sources of information about society 
A student who has taken courses in history and other social 
studies is expected to know how to And and use intelligently 
the various sources of information relating to current society. 

Such courses include: 

Current gossip — oral and printed. 

Reasoned discussion — oral and printed. 

Social activities — real or pictured. 

Material achievement — ^in use or relic. 

The skills necessary to deal with such sources range from 
simple physical findings and manipulation to subtle c ritical 
analysis. A series of tests ranging from the simple to the more 
complex phases of these skills must be constructed. Some are 
already available.*' 

H. E. Wlson of Harvard University collaborated with the staflF 
of the Investigation in trying to formulate a check list of '’skills" 
or "working habits” as those were recognized by teachers. One 
of his students, Price, conducted a questionnaire inquiry of teach- 
ers, the tabular results of which are here set forth. 

It was early apparent that so little work had been done in the 
measurement of such skills as the social studies required that only 
pioneer wcu'k could be attempted by this Investigation. The work 
of Parker and Calkins, directed primarily at the testing of geo- 
graphical knowledge and understanding, promised to blaze a wide 
&ail in& the field of graphic skills. Marion Dark and E. M. Hunt 
nlMtertqpk to test the less graphic skills associated with critical 
rmdinil^ Clark succeeded in devising some elementary tests of 
hucoridid criticism whidi ue here desoibed. 
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REPORT OF THE WORK OF THE COMMITTEE ON 
GEOGRAPHY TESTS' 


Edith Putnam Pakjcsr 
Ij^niversity of Chicago 

Inotal Discussion and Proposals 

In 1930, members of a proposed geography committee were in- 
vited to confer with representatives of the Commission for the 
Investigation of the Social Studies in the Schools about the advis- 
ability and feasibility of testing, under the auspices of the Com- 
mission, for fundamental geographic attainments believed to be 
of value in the pursuit of social studies. The advisability of so 
doing hinged in part on the relation of geography to the social 
studies. Modern geography is a science which may be termed 
‘‘socio-naturar’ to distinguish it from the social studies and from 
the natural sciences. Ifthe testingworkof the Commission was to be 
restricted to the purely social studies, tests in geography obviously 
could not be included legitimately. All participants in the discus- 
sion recognized, however, that the progress of pupils in the social 
studies probably is facilitated by the possession of various geo- 
graphic understandings, abilities, and attitudes; they thought, ac- 
cordingly, that data of value to the Commission and not available 
from other sources could be secured by the use of geography tests 
focused on such attainments. The geographers realized that the 
sponsorship of the proposed tests by a Commission on Social 
Studies might give the false impression that geography was being 
induded as a social study rather than as a science that contrihuUSy 
among other things, to intelligent, efective pursuit of social studies. 
On the other hand, they appreciated and welcomed the opportu- 
nity to be of service, and realized th^ experimentation with sudi 
tests should be of value also in improving the teaching of grogra- 
phy. In view of the latter considerations, it was decided to risk 
the possibility of misimpression noted, and to undertake the task, 
if> up(m other grounds, the project seemed desirable and feasible. 

Oednon with regard to the desirability of the undertaking 
The teits are published by Charles Scribner’ s Sons. 
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hinged on the quegtion, **May the teits be concerned wholly with 
basic i>al%us inherent in sound geographic training?** It is recognized 
widely that, in spice of recent improvement in the teaching of 
gec^raphy in the schools, actual procedure in many geography 
classes centers attention largely on the memorization of miscel- 
laneous facts. It fails to lead pupils to derive, from carefully 
guided use of relevant facts, many of the fundamental values of 
geographic training. Obviously, then, testing for evidences that 
those basic values had been derived would not be synonymous 
with testing for what had been accomplished in the schools under 
the flag of geography. It would show only what had been accom- 
plished in the way of fundamental geographic outcomes. 

The geographers involved made it clear that they had no interest 
in measuring progress along lines not directed toward basic geo- 
graphic goals. They expressed the conviction that tests designed 
for measuring so-called geographic progress, instead of basic geo- 
graphic progress, not only yield findings of little or no value, but 
also tend materially to retard improvement in the teaching of 
geography, and in the services that it may render to the social 
studies. They called attention to the general belief that an achieve- 
ment test indicates what experts think should be achieved, and to 
the fact that teachers accordingly tend to set up standards of 
achievement in terms of the abilities and understandings which 
their children need in order to make a reasonable showing on 
achievement tests. They submitted that improvement in geo- 
graphic training can be brought about through tests only to 
the extent to which testing emphasis is placed on fundamental 
g 03 «a^ic achievements. 

VteNqng the matter fflbin a positive rather than a negative angle, 
they darned that tests streadj^ geographic essentials would serve 
to increase an understandmg|l(^xhe part of the educational public, 
of iralues that are inherent in geography, and would tend to stimu- 
late ^and ^piide eflTorts to derive those values. 

Tne; pipjact, then, seemed undesirable unless the tests could 
ttresa |K^devements resulting from the most valuable type of 
g|Qgra|»fatc ironing. 
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The qumion of feasibility also hinged largely on this issue of 
testing for “what ought to be*’ rather than for “what is.” Even if 
it were desirable to discover what is being done, regardless of its 
value, such a task would have been impracticable in view of the 
necessary restrictions as to length and number of the tests. The facts 
that comprise most courses in “geography” are exceedingly numer- 
ous and varied. General testing experts long have been baffled by 
the problem of formulating brief, comprehensive tests in cases 
where facts are multitudinous and of many types. Pupils in class 

A, with as much fact baggage as those in class B, or even more, 
may make a much lower score than the latter on a given “fact 
test” because the particular facts chosen were not in their reper- 
toire. With another sampling, class A might seem superior to class 

B. It is clear that sound conclusions concerning what is being 
achieved in “fact courses” can be based only on many and varied 
samplings of each representative group of pupils tested. 

The difficulty disappears in large measure if tests center on major 
understandings, abilities, and attitudes to be derived from the use 
of relevant facts, for the fundamental attainments of these types 
are not so numerous as to preclude effective sampling in relatively 
brief tests. In such a case, one tests for many facts, as the later 
discussion will show, but he tests for them not as ends in them- 
selves but as elements in abilities and understandings. He tests 
for power to use facts effectively in the types of situations in which 
their use is helpful. For example, a test which involves reading 
facts of various types from a given physical map puts emphasis on 
a functioning ability. It samples adequately one’s ability to read 
other maps of similar kind, and the elements in map-reading ability 
are few enough so that eadt can be touch^ upon in a relxdvdy 
brief test exercise. A pupil needs to inow many facts in order t© 
read such a map well. Evidence of^ility to read the map thus is 
evidence also of possession of these facts. Evidence of lack of diis 
ability, or of particular phases of it, is evidence also of (1) lack of 
the necessary information concerning basic facts, or (2) inability 
to make that information function in the practical situation in* 
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vdved. The same general points may be made in connection ^itb 
any type of major geographic understanding or ability. 

As a result of the discussion along the foregoing lines, the repre- 
sentatives of the Commission proposed: 

(1) that it be made clear in the report of the Commission that 
gec^raphy was involved not as a social study, but as one 
which nas contributions to make to the pursuit of social 
studies; 

(2) that a geography committee construct experimental tests 
of achievements which represent basic geographic attain- 
ments; 

(3) that the tests be not limited to what is achieved generally, 
but that they be based on what it is practicable to achieve 
through the best type of geographic training which can 
be given; and 

(4) that the tests be objective, comprehensive, group tests 
planned for use with college freshmen. 

>To understand the reasons for the nature of the tests constructed 
and for the handling of the data secured through their use, it is 
essential to bear in mind the special type of test which was speci- 
fied in this agreement. 

Mr. R D Calkins, of the Department of Geography, State 
Teachers College, Mount Pleasant, Michigan, and Miss Edith 
Parker, of the University of Chicago, were asked to cooperate with 
Dr. Bowman in the undertaking. The Commission felt that the 
expenences of these three geographers would enable them to make 
diversified contributions to the work of the committee. 


Preliminary Work and Plans 
* 

In planning the procedure to be followed in fulfilling the assign- 
mieat, committee recognized the necessity of (1) reviewing 
atiidiei PTzady made coheeming the fundamental values to be 
Sit^nievdl thrmil^ g^>graphic training, (2) comparing the findings 
1^ with the beliefs of professional geographers as ex- 
p^paaiMdliir n^ddbnced in texts and in other published or impul^shed 
ottdining the types of understandings and abilkies 
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which, according to the results of (1) and (2) probably should be 
tested for, and (4) securing the reactions of others to that outline. 

A series of studies, \ater reported in the 1933 Yearbook of the 
National Society for the Study of Education, had shed much light 
on the problem of selecting major types of achievement which are 
distinctively geographic. The technique used in those investiga- 
tions had precluded the possibility of favoring any particular de- 
limitation or definition of the subject. All types of materials called 
“geographic” by their authors had been analyzed, and, by the 
comparison of the types of ideas and abilities developed thereby 
with types of ideas and abilities developed in materials not labelled 
geography, those distinctive in geography had been ascertained. 
There was some disagreement among members of the geography 
committee itself as to the proper scope of the subject and as to the 
terms that best describe some of the distinctive types of ideas 
developed therein, but these differences were of little or no signifi- 
cance in the task at hand. To define geography, to settle 
differences of opinion concerning its terminology, and to attempt 
a delimitation of its scope were not among the assigned duties. 
It was obvious from the reviews of studies, of texts, and of opinions 
of experts that there existed a body of fundamental, distinctive 
understandings and a group of abilities needed in acquiring them 
effectively, upon which essentially all geographers seemed agreed, 
regardless of their definitions of the subject, and of their favorite 
terms for describing these types of outcomes. The committee ac- 
cordingly adopted the policy of centering attention^on^those abili- 
ties and understandings upon which all seemed to be in accord. 
The ability to read maps for geographic purposes, for example, 
was so conspicuously a pai^ of every prbposed program of geo- 
graphic training, was so emphasized in every text in the subject, 
and was so clearly envisaged by every exposition or philosophical 
discussion of geography that it seemed altogether safe to list vari- 
ous phases of the ability to read maps among those which would 
be sanctioned by all geographers as representing one of the basic 
values of the study. The major types of understandings and abili- 
ties listed were: 
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Understandings 

1. Of the interrelations of cultural and natural features which 
characterize specific regions, such as France, western United 
States, a given desert region in low latitude, and the like. 

2. Of the nature, conduct, and world distribution of given 
kinds of human activities {e. g., the manufacture of rubber 
goods) as related, on the one hand, to the cultural assets of 
the people engaged in them and, on the other hand, to the 
natural factors these people capitalized or coped with in 
carrying them on. 

3. Of various ways in which peoples of different cultural as- 
sets capitalize or cope with specific types of natural factors, 
such as climate, topography, and the like. 

Abilities 
The ability: 

1. To read from landscapes, pictured or real, facts of signifi- 
cance in acquiring these understandings. 

2. To read such facts from maps. 

J. To read such facts from graphs and statistical material. 

4. To acquire such facts from the printed pag^ (verbal reading 
matter). 

5. To reason accurately in terms of such facts. 

6. To express effectively geographic information and geo- 
graphic conclusions based on such reasoning. 

7. To apply to new situations understandings and abilities of 
the types indicated. 

Practically all authors of texts and of courses of study in geog- 
raphy for elementary schools, state, in essence, that one objective 
of the work is to give understandings of the first general type 
named. Understandings of the second type are stated, in effect, 
to be a goal of work outlined in texts and courses for secondary 
sdiools. I^ch courses commonly are designated by the terms 
”commer^al,” “economic,” or “political” geography. In courses 
of study commonly called "principles of geography,” “elements of 
gepgenph^,” “phj^ical geography,” and the like, and in texts used 
in llwm, the giving of the third general type of understanding 
cuiiontaity is evidenced as an objective. In connection with the 
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gaining of any of these types of understandings, texts and courses of 
study stress the desirability of developing abilities of the various 
types noted. In so far as their expressed purposes are concerned, 
authors of these guide materials all seem to seek these fundamental 
achievements. Obviously the general emphasis in current schooh 
room practice on the mere memorization of facts is not in harmony 
with the avowed purpose of experts in the field. It seemed clear 
from the expressions of purpose examined that the list represented 
types of achievements that at least the vast majority of geogra- 
phers would agree upon as of fundamental importance. 

Dr. Bowman, who had conducted a separate survey of opinions 
for a different purpose, felt, on the basis of his findings, that this 
was the case. To secure a further check, the committee, in con- 
ference with representatives of the Commission, listed geographers 
at fifteen centers of geographic training as those to whom the list 
should be sent first for criticisms. It was felt that the group selected 
represented a fair sampling of expert geographers and that, if they 
offered no objections to the list, the committee could be reasonably 
certain that it was a sound one on which to proceed. In the letter 
accompanying the list, the geographers addressed were asked to 
express freely their opinion concerning the advisability of testing 
for attainments of the types indicated rather than for mere facts 
as ends in themselves. Every reply sanctioned the former pro- 
cedure and condemned the latter. No objections were raised as to 
the types of attainments listed. Two queries were made about 
terms used in describing the understandings, but it was made clear 
in each case that there was no objection to the essence of the thing 
expressed. 

As a result of this inquiry, the committee felt reasonably sure 
that to base a test on the understandings and abilities listed would 
meet with the general approval of professional geographers. It is 
significant, as bearing out that conclusion, that as the tests them- 
selves were presented later, all of the many geographic critics who 
reacted to them seemed to accept the attainments involved as 
unquestionably of basic value. 

It next was decided to make a test consisting of several ‘^parts’* ^ 
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and to devote one part to each of the three major types of under- 
standings listed. Part I, for example, was to be devoted to testing 
for understandings of the first type named. It was obvious, how- 
ever,^ that it would be impracticable to devote a separate part to 
each t3npe of ability indicated. Experimentation showed that 
graphs and pictures could be used in testing for some of the under- 
standings. TTius, a person who connected with a specific region 
facts shown by a group of graphs would evidence not only a knowl- 
edge of that region but also an ability to read significant items 
from those graphs. It was realized that an incorrect reaction to 
such an exercise might be due to lack of knowledge of the region, 
to lack of ability to read the graphs accurately, or to both. Some 
light would be thrown on the nature of the difficulty, however, by 
comparing this reaction with others. Map reading, unlike land- 
scape and graph reading, involves so many phases of ability that 
it was impossible to test adequately for ability in it in connection 
with testing for understandings. In view of this fact and of the 
importance of developing these various phases of the ability, it was 
decided to devote a separate part of the test to map reading. As 
one basis for the diagnosis of failures evidenced in connection with 
the other four parts, it was planned to devote a fifth part to a vo- 
cabulary test dealing with technical and semi-technical terms. 
The general plan of the test may be summarized as follows: 

Part I. World understanding in terms of understandings of 
regions commonly introduced in elementary geography: 
some of the exercises to be based on pictures and graphs. 

Part II. Map reading ability: exercises to be based on accom- 
panying map book. 

Part III. World understanding in terms of understandings of 
actitities commonly treated in secondary school geography: 
smtie of the exercises to be based on pictures and graphs. 

Part" IV. World ' understanding in terms of understandings 
dev^oped^ in high school courses of the types noted in con- 
nection ^h the third major type of understandings: some of 
the exercises to be based on pictures and graphs. 

PartV. Vocabulary test. 
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The Construction of the Test 

It was decided to construct first the part of the test dealing with 
map reading. This proc^ure would make it possible to carry for> 
ward work on the map books which were to accompany the exer- 
cises while other parts of the test were being prepared. Moreover, 
it seemed probable that experience with this part, which could be 
completed with less difficulty than any other, would facilitate the 
construction of the other parts. Selection of items was relatively 
simple because it hinged upon such definitely ascertainable facts 
as (1) the types of maps for which one has need in everyday affairs, 
(2) the types of symbols used on such maps, and (3) the types of 
ideas which are expressed in terms of these symbols. Examination 
of current texts, newspapers, periodicals, atlases, and maps indi- 
cated that the types of maps used most are (1) black and white 
“sketch’' maps, consisting of symbols for such features as cities, 
railroads, rivers, and mountains; (2) automobile road maps; (3) 
political maps; (4) physical, or color-contour maps; (5) maps de- 
picting such items as forest areas, wheat and other crop acreage, 
and rainfall; and (6) weather maps, topographic maps, and the 
like. The symbols used on these maps are of four major types, 
namely: (1) semi-pictorial signs; (2) color bands, color spots, and 
dots; (3) orientation lines — the network of parallels and meridi- 
ans; and (4) “iso” lines. 

Semi-pictorial symbols are those that suggest by their form the 
features they symbolize. The symbol for a coast line, for example, 
suggests the appearance of an actual coast line as seen from the air. 
Practically all symbols on sketch maps and automobile road maps, 
many of those on political and physical maps, and some of those 
on nearly all maps are of this type. Because they partake some- 
what of the nature of a picture or drawing, they are the simplest 
type of symbol to read. In the accurate interpretation of symbols 
of the second type named, careful use of accompanying legends is 
required because these symbols are not semi-pictorial. Dots may 
stand, for example, for a given acreage of wheat on one map, for a 
designated production of rye on another, and for a given number 
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of sheept or for something entirely different, on a third. Accord- 
ingly, the interpretation of these symbols is less simple than that 
of symbols of the semi-pictorial type. The colors on political maps 
and on color-contour maps, and the colors, shades, or dots on 
various distribution maps are of this second type. Orientation 
lines may be used on any type of map, and they are found almost 
invariably on the political and physical maps in common use. The 
translation of line symbols of both the orientation type and the 
iso-line type into the conditions they depict demands a higher 
order of ability than that required for the interpretation of sym- 
bols of the first two types indicated. 

The kinds of ideas to be read from semi-pictorial symbols are 
those of (1) the naturt of the features symbolized (including their 
size, extent, and shape, as well as their general type) and (2) the 
relative location of particular features with regard to other features 
(direction and comparative distance). The chief function of color 
bands, color spots, and dots is to give ideas concerning the areal 
distribution of the features, conditions, or activities represented. 
The purpose of orientation lines is to give (1) ideas of distance and 
direction from the equator and prime meridian^ and (2) ideas of the 
relation of any part of the earth's surface to other parts of its surface 
and to its surface as a whole, in terms of size and location. Iso-lines 
convey (1) ideas of exact elevation, slope, air pressure, wind direction, 
temperatures, and the like, and (2) ideas of the areal distribution of 
the conditions they depict. 

In making use of the foregoing groups of facts, the committee 
based its work on the following considerations: 

1. However valuable a map may be for use in everyday affairs, 
it is useful to a given individual only to the extent to which he can 
get from it the ideas it is designed to convey. 

2. It follows that testing for map-reading ability of a type which 
it ts desii|Lble to develop in the course of elementary and secondary 
odu^tiMi means testing for the ability to get from maps, of types 
^at funoi^n helpfully in everyday affairs, the ideas those maps 
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3. In accordance with the initial agreement concerning the 
nature of the test, this part would be based on map-reading ability 
needed in everyday affairs and not on a consideration of that 
possessed by most graduates of secondary schools. 

4. Findings from the use of such a test probably would show 
that many, or even most, of the persons tested possessed only a 
few phases of this ability. Such a result would not point, however, 
to the impracticability or undesirability of developing the other 
phases, but merely to the fact that the experiences of those tested 
had not been such as to Insure that development. Until one knows 
what phases of map-reading ability have resulted from the experi- 
ences pupils have had, and what phases of it have not resulted, 
one has no sound basis for providing more helpful experiences. 
Only a test constructed on the basis of what constitutes desirable 
map-reading ability, then, would be of value in a program of 
improvement. 

In planning the map book to accompany the exercises, it obvi- 
ously was desirable to provide, in so far as practicable, for maps of 
each of the several types above listed. The expense of colored 
maps was prohibitive, but it proved feasible to include all other 
types. 

The necessity for brevity led to the construction of exercises of 
a true-false type, and to the inclusion in many cases of only one 
exercise for a given phase of ability. It is to be noted, however, that 
none of the true-false statements was such as to call for mere mem- 
ory. All of them were based on the accompanying maps, and one 
can judge of their truth or falsity only in terms of what he reads 
from the maps to which they refer. 

As a basis for checking the relative effectiveness of these true- 
false exercises, multiple choice exercises dealing with the sami^ 
phases of ability were devised. Two forms of true-false and two 
forms of multiple choice tests were constructed. In each form, 
exercises were so grouped as to facilitate the use of the accompany- 
ing map books. 'Bie arrangement was such that the turning of 
pages, for example, was minimized. Grouping also was such as to 
accord with standard chance distribution theorems. In the true- 
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false forms, approximately fifty per cent of the exercises were fol- 
lowed by those calling for reactions of the opposite type; twenty- 
five per cent were followed by two exercises calling for reactions of 
the opposite type; twelve and a half per cent, by three opposites; 
six and a fourth per cent, by four opposites, and six and a fourth 
per cent, by five or more opposites. The number of “trues” 
equalled, of course, the number of “falses.^* In the multiple choice 
forms, care was taken to group the statements from which selec- 
tions were to be made so that the first statement was correct in 
approximately twenty-five per cent of the cases, the second in ap- 
proximately twenty-five per cent of the cases, and so on. They 
also were arranged to avoid any set order: an exercise in which 
the first statement was correct was followed in some cases by one 
in which the first statement also was correct, and in other cases by 
one in which the second, third or fourth statement was correct. 

In the course of the work on the map test, preliminary experi- 
mentation was carried on with a group of about two hundred 
and seventy students in the Teachers College at Mount Pleasant, 
Michigan. The chief purpose was to ascertain the amount of time 
needed to react to various types of exercises. The conclusion was 
reached that the test must be limited to about fifty exercises, to 
meet the time requirements set by the Commission. It was decided 
to include exactly fifty items, each to be given one score point, in 
the belief that this procedure would minimize work with scores. 

Before rfiese tests, designed wholly for college freshmen, were 
completed, the committee was told of the intention of the Com- 
mission to use the tests also in all grades from the fourth to the 
twelfth inclusive. This intention did not introduce any new factor 
into the work save that it led to a reconsideration of the language 
used with a view to making it as simple as possible. All the other 
considerations on which the work was based held for the testing 
of pupils throughout the grades indicated. Any test which would 
serve to show what phases of map-reading ability college freshmen 
possessed and what phases they did not i>ossess would serve to 
llnisr the same things in the case of pupils in any lower group who 
could lumilb the terms used. 
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The following table indicates the specific phases of map-reading 
ability covered, and the numbers of the questions in each form 
which were designed to test the respective phases. 


PART II 

TESTING THE ABILITY TO READ MAPS 
Section A. Using Semi-Pictoeiai. Maf Symbols 


Specific Abilities Tested 

Exercises Used for Testing 

Form I Form II Form I Form II 
T. F. T. F. M. C. M. C. 

Abilip' to use semi-pictorial map symbols 
to identify: 

sea and land 

1 

2 

1 

1 

railroad 

2 


, 

2 

river 


1 

, , 

, * 

strait 

3 


3 


mountainous land 



2 


cities and mountainous land 

4 



, , 

cities and coast 


3 



cities and rivers 

. , 



4 

cities and lowland 

, , 

. , 

4 


island 




3 

peninsula 

. • 

4 

. • 

. . 


Ability to get from semi-pictorial symbols 
ideas of; 


general direcuon 

5 

5 

5 

S 

direction of stream flow. 

6 

6 

6 

6 

relation between rail route and river 
valley 

7 

8 

7 

7 

comparative distance (without use of 
scale of miles) 

8 

7 

8 

8 

Ability to use scale of miles in determining 
distance 

9 

9 

9 

9 

Ability to use ideas of relations between 
street patterns and natural conditions 
in interpreting facts shown about 
street patterns 

10 

10 

10 

10 

Ability to use symbols and legend on 
convenrional automobile road map to 
determine distance, direction, surfac- 
ing, and highway marlcing 

11 & 12 

U&12 

11 & 12 

11 k 
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Sectioh B. Using Color Band, Color Spot, and Dot Syubols 


Exercises Used for Testing 

Specific Abilities Tested Form I Form II Form I Form II 

T. F. T. F. M. C. M, C. 


Ability to use such symbols in reading 
facts concerning: 

elevation 

density of population.. 

Ability to use such symbols in comparing 
facts about: 

density of population and amount of 

rainfall 

elevations and rainfall 

Ability to see probable relationships be- 
tween facts read about: 
production of crops and amount of 

rainfall 

trade, production of crops, and den- 
sity of population 

topography, wind directions, and 

rainfall 

temperature and altitude 



14 

14 

• • 

IS 



IS 

. . 

15 

15 

* • 

16 



16 


is 

16 

• • 


Section C. Using Orientation Lines 


(The Network of Parallels and Meridians) 

Exercises Used for Testing 


Specific Abilities Tested 


Ability to use meridians and parallels in 
reading directions from a network in 
which meridians and parallels are 
straight lines; 

north-south 

east-west. 

Ability to use meridians and parallels in 
reading directions from a network on 
which meridians and parallels are 
curved lines: 

east- west 

north-south 

northeast-southwest 


Form I Form 11 Form I Form II 
T. F. T. F. M. C. M. C. 


18 

19 & 20 


ibflity to read from a strai^t-line net- 
worc facts «bout: 

idimsion^roin equator 21 

fayfearfkai terms of degrees 22 
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Sbction C. Usimg Oubntatiom Lines— C oiKinti/i 


(The Network of Parallels and Meridians) 

Exercises Used for Testing 


Specific Abilities Tes'’ted 

Form I 
T. F. 

Form II 
T. F. 

Form I 
M. C. 

Form II 
M. C. 

latitude in terms of high, low or 





middle 

latitude in terms of comparison with 

24 



19 

distance from equator to pole 

degrees of latitude as translated into 


22 


22 

miles 

Ability to use facts thus read about lati- 
tude in explaining: 


24 

22 


type of vegetation 

23 


, , 

21 

length of growing season 


23 

21 


length of days 

Ability to read from a curved-line net- 
work: 




30 

latitude in terms of degrees 

latitude in terms of nigh, low or 

25 

27 



middle 

latitude in terms of comparison with 


26 

33 

• ■ 

distance from equator to pole 

degrees of latitude as translated into 

27 


24 

• • 

miles - 

Ability to use facts thus read about lati- 
tude in explaining: 

28 

25 

* » 

26 

type of crop grown 

26 

25 

25 

length of growing season 

. . 


length of day 

Ability to read from a straight-line net- 
work facts about: 


28 

26 


direction from prime meridian 

29 


, , 


longitude in terms of degrees 

Ability to use facts thus read in: 

30 


28 

■ * 

estimating distance in miles 

. . 

. . 

. . 

23 

comparing time 

Ability to read from a curved-line net- 
work facts about: 




29 

direction from prime meridian 


29 

27 


longitude in terms of degrees 

longitude in terms of comparison 
with distance from prime meridian 

• * 

30 

• ' 

24, 28 

to 180th meridian 

31 

• • 


27 
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SscnoN C. Usivo Omkmtation Lnccs — Conttimed 


(The Network of Parallel* and Meridian*) A 

Exerdte* U*^ for Testing 


Specific Abilities Tested 

Form I 
T. F. 

Form II 
T. F. 

Form I 
M. C. 

Form 11 
M. C. 

AbSity to UM facts thus read in: 

estimating distance 


31 

23, 29 


comparing time 

32 

32 

30 

• • 

Ability to compare extent of an area 
shown on straight-line network with 
extent of one shown on curved-line net- 
work 

33 

33 

31 

31 

Ability to identify map of given region on 
one network with same region on an- 
other network 

34 

34 

32 

32 

Ability to explain seeming differences in 
the shape of a given region on two maps 




33 

Ability to use straight-line network in 
reading: 

difference in latitude of two given 

places • • . 

difference in longitude of two given 
places 

35 

35 

35 

35 

36 

36 

34 

* • 

difference in size of areas extending 
through the same number of de- 
mes of longitude and latitude, 
but in different latitudes 

38 

38 

36 

34 

Ability to identify given latitude and 
longitude with specific continents and 
oceans 

37 

37 

35 

36 

Ability to: 

read directions from a polar projec- 
tion 

39 

39 

37 

37,38 

identify meridians and parallels on a 
polar projection 

40 

40 

38 

39 


Section D. Using Iso-Lines or Various Types 

(Isotherms, Isobars, and Contours) 

Exercises Used for Testing 

Specific Abilities Tested 

Form I Form II 
T. F. T. F. 

Form I Form II 
M. C. M. C. 

to use isothermal maps in reading 
Mid eoE^ibig fads about; 
nvffive monthly tempmtuses at 
JNW or mote places in a given 

41 41 

39 
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Section D, Using Iso-Limsb of Vakious Tvfes 


(ttOTHERMS, IsOBAM, AND CoNTOUM) 

Exercuet Used in Testing 


Specific Abditiet T^ed 

Form I 
T.F. 

Form n 
T. F. 

• Form I 
M.C. 

Formli 

M.C. 

range in temperature between two 
sets of places 

42 

42 

• • 

40 

average temperature for two or 
more given months 

43 

43 

41 

41 

average temperature in a given 
month and latitude 


• • 

40 

* * 

range in temperature and location. . 


• • 


42 

Ability to use facts thus read in explaining 
crop productions 

44 

44 

42 

• * 

Ability to use isobaric maps in reading 
facts about; 

pressure and wind direction 

45 

45 

43 

44 

pressure and wind velocity 

• • 

• * 

• • 

43 

Ability to use facts thus read in inter- 
preting weather conditions 

46,47 

46,47 

44, 45,46 

45,46 

Ability to use contour maps in: 

reading facts about elevation 

ii 

4g 

47 

49 

identifying topographic profiles 

50 

50 

SO 

• • 

identifying topographic features, . . . 

• • 

t « 

49 

a * 

Ability to use facts thus read in helping 
to explain: 

choice of routes 

49 


48 

• • 

selection of town sites 


49 

• « 

« • 

flood hazards 

• • 

• • 

• 4 

50 

Ability to identify isobars, isothenns, 
isohyets, and contours 

• • 

• • 

• • 

47,48 


Though Part II was the least difficult to construct, much time 
was consumed by the experimentation at Mount Pleasant, the 
analysis of its results from various angles, the construction of the 
maps for the map book, and the reworking of the tests to meet 
(I) the various conditions outlined above and (2) minor defects in 
wording revealed by critical study of the exercises and informal 
experimentation with them. In the four months the committee 
could devote wholly to the work, the map tests were completed, 
Part I was planned in detail, and the remaining parts were blocked 
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out roughly. It was impossible after that time for any two mem- 
bers of the committee to work in the same place, and the necessity 
of conducting the work by means of correspondence slowed it 
materially. Part I was completed in this way, but little more was 
done with the last three parts. General plans made for them will 
be discussed in a later section of the report. 

The procedure in preparing Part I of the test was similar to that 
described in detail for Part II. The purpose, as noted earlier, was 
to test for an understanding of the world in terms of a knowledge 
of outstanding characteristics of its various regions. Since some of 
the characteristics of any region are associated closely with its 
latitude, several exercises were devoted to testing for the ability 
to read from landscapes, descriptions, and graphs, features or facts 
indicative of latitude; and for the ability correctly to associate 
those features or facts with such location. For example, “The 
places shown in Figures 1 and 2 [pictures] are probably about the 
same distance from the equator.” Correct reaction involves a 
knowledge of the types of vegetation typical of low lattitudes and 
of middle latitudes, and an ability to recognize those types in 
terms of the broad-leafed (banana, etc.) trees shown in one picture, 
and the grass lands shown in the other. 

“The region [described in the preceding paragraph] is near the 
south pole.” In that paragraph, facts are stated which are true 
only of places in high south latitude. Knowledge of conditions near 
the south pole enables one to react correctly. 

“Since June is one of the cooler months there, the country is 
south of the equator.” The statement refers to a temperature 
graph. If one reads the graph correctly he finds the statement 
false because it shows June is one of the warmer months in the 
region in question. Ability to read the graph, and knowledge of 
the time of occurrence of seasons north of the equator and south 
dP it enable one to react correctly. 

The exenases in Part I which deal with this general ability are 
listed bdowi The second column indicates the type of the source 
from wirndi facts were to be read, and the third column the types 
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of association or relationship, knowledge of which is needed to 
make correct reactions. 

Group I. Recognizing signs of location with regard to the 
equator and associating them properly with such location. 


Source of sig- 
Item nificant facts 

Types of associations involved 

1 

picture 

nature of vegetation 

relative distance from equator 

3 

picture 

nature of vegetation 

conditions in northern and 
southern hemispheres. 

4 

picture 

nature of vegetation 

conditions in northern and 
southern hemispheres 

22 

description 

continuous daylight in De- 
cember 

distance and direction from 
equator 

S2 

description 

nature of crops 

distance from equator 

59 

description 

nature of crops and growing 
season 

direction from equator 

60 

description 

nature of crops 

distance from equator 

61 

description 

nature of crops and growing 
season 

extent in latitude 

76 

graph 

time of occurrence of seasons. 

direction from equator 


About three-fourths of Part I tested for the ability to identify 
with specific regions given groups of facts or relationships which 
are outstanding characteristics of the regions in question. Great 
stress was put on this phase of world understanding for the follow- 
ing reasons: 

1. It is the alleged purpose of elementary instruction in geog- 
raphy to give pupils correct impressions of the modes of life in 
various regions of the world, and to help them properly to asso- 
ciate human activities characterizing these regions with natural 
conditions typical there. 

2. If one has learned the geographic characteristics of a country, 
he should have no difficulty in identifying them when they are de- 
picted in landscapes, in descriptions, or in graphic representation. 

3. Of the various achievements that result from sound geo- 
graphic training, none is more useful in contributing to progi^ss in 
the sodal studies than a knowledge of the geographic personalities 
of various regions of the world. 
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Care was taken to select regional characteristics of importance 
and to deal only with regions or places frequently referred to in 
current literature and treated in practically all elementary texts. 

The test items devoted to this ability to identify the chief geo- 
graphic characteristics of various countries or places were of two 
kinds. One of them may be thought of as constituting Group II 
and the other as constituting Group III. In items in Group II, the 
characteristics to be rcfentihed were shown in pictures, described 
verbally or depicted in graphic form. The persons tested were 
asked tell whether the facts thus presented fit given regions or 
places, or do not. In items in Group III, the matching was in 
reverse order. A given region or place was named, and various 
facts or relationships lifted. In the case of each fact or relationship 
those tested were asked to tell whether or not it helps to character- 
ize the region ill question. In items of the Group III type, several 
reactions are called for in the case of some of the countries or places 
to be characterized. The time to which administration of the test 
to a given group had to be limited made it impossible to treat 
many regions in this w^y. 

Examples of exercises of the Group II type follow, 

“There are places much like the one shown in Figure 1 [a picture] 
in central Africa.” 

“This description fits many places in the Sahara and south- 
western Asia.” 

“All facts shown [in the accompanying graphs] are true of 

Egypt.” 

TTie fbffowmg fists indicate the items in Group II, the types of 
sources from which characteristics were to be read in each case, 
the types of characteristics to be associated with the regions in 
ijuestion, md the regions with which they were to be identified. 

Group II. Identifying given facts or groups of facts or relation- 
ahips with specific regions or places. 

Source of sig- 

lum iiiScant fact* Types of associations involved 

^1 " < 1 lilI j i M i l l i I I ■ _ 

10 puatiiOT . . ■ . ■ ■ nature of vegetation . . southernmost North America 
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Source ot sig- 
Item niiicant facts 

Types of associations involved 

11 

picture 

nature uf vegetation 

southernmost South America 
and Australia 

12 

picture . 

nature of vegetation . . 

northernmost Africa 

13 

picture. . 

nature of vegetation. . . 

central Africa 

14 

picture. , 

nature of vegetation . . 

southern Europe 

IS 

picture . 

nature of vegetation . 

northern South America 

16 

picture . . . 

vegetation, surface, and hu- 
man activity. . . 

the Nile region 

17 

picture . . 

vegetation, surface, and hu- 
man activity.. . 

the Congo region 

18 

picture . . 

vegetation, surface, and hu- 
man activity. . . 

the Amazon region 

19 

picture , . 

vegetation, surface, and hu- 
man activity . . 

Switzerland 

20 

picture . . 

vegetation, surface, and hu- 
man activity 

Norway 

21 

picture 

vegetation, surface, and hu- 
man activity . . . 

Holland 

25 

description. . . 

nature of houses, food, and 
work .. 

Sahara and southwestern Asia 

26 

description . . 

nature of houses, food, and 
work 

southwestern United States 

30 

description. . . 

nature and distribution of 
work, crops, and vegetation 

northern United States and 
southern United States 

31 

description. . . 

nature and distribution of 
work, crops, and vegetation 

eastern United States and 
western United States 

32 

description. . . 

nature and distribution of 
work, crops, and vegetation 

Virginia, Kentucky, Missouri, 
and Kansas 

33 

description. . . 

nature of work and location in 
given part of the United 
States. . . 

Lowell [Massachusetts] 

34 

description. . . 

nature of work and location in 
0 ven part of the United 
States.. 

Atlanta [Georgia] 

35 

description. . . 

nature of work and location in 
given part of the United 
States 

Birmingham [Alabama] 

36 

description.. . 

nature of work and location in 
given part of the United 
States 

Pittsburgh [Pennsylvania] 

38 

description. . . 

nature of farms, of farm crops, 
and of farm work 

New England 
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Source of ng- 

Iteoi luficant facta Types of associations involved 

39 

description. 

. . nature of farms, of farm crops, 
and of farm work. . 

States bordering Great Lakes 
on the south 

h 

description. 

nature of farms, of farm crops, 
and of farm work 

States at eastern base of the 
Rockv Mountains 

41 

desciiption 

nature of vegetation and tem- 
peratures 

northwestern United States 

42 

"Sescnption. . 

. . nature of veeetation and 
amount and distnbution of 
rainfall 

northwestern United States 

44 

description . 

. kinds and distribution of work 

Washington [state] 

45 

description. . 

. kinds and distribution of work 

Oregon 

46 

description. . 

kinds and distribution of work 

North Carolina 

53 

description. . 

kinds and distributionofwork, 
density and movement of 
population 

Chile 

54 

descnption. . 

. kinds and distribution of work, 
density and movements of 
population 

Italy 

55 

description.. 

. kindsanddistribuuonofwork, 
density and movements of 
population 

Spain 

56 

descnption 

distribution of populauon and 
railroads; important prod- 
ucts and exports 

Russia 

57 

descnption^ 

distnbution of population and 
railroads, important prod- 
ucts and exports 

Canada 

58 

descnption . 

distribution of population and 
railroads, important prod- 
ucts and exports 

Australia 

62 

description .. 

surface, climate, density and 
distribution of population, 
standard of living, mineral 
resources, nature and dis- 
tribution of work. 

China 

63 

description.. . 

surface, 'kdimate, density and 
distribution of population, 
standard of living, mineral 
tesourtts, nature and dis- 
tribution of work., . . 

India 
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Source of 

Item nificant facta Types of associations involved 

64 description. . . surface, climate, density and 
distribution of population, 
standard of living, mineral 
resources, nature and distri- 

butionofwork Brazil 

78 graphs climate, land utilization, crop 

bar acreage, imports and ex- 
line ports Union of South Africa 

pjg 

79 graphs climate, land utilization, crop 

bar acreage, imports and ex- 

line ports Egypt 

pie 

84 description.., location; nature of tributary 

area Buenos Aires 

85 description... location; nature of tributary 

area Calcutta 

86 description... location; nature of tributary 

area Marseilles 

87 description. . , location; nature of tributary 

area Cairo 

88 description... location; nature of tributary 

area Sydney 

Examples of exercises of the Group III type and list of items in- 
cluded in that group follow: 

(These statements were to be evaluated as true or false.) 
^‘Chicago, in part because it lies in a vast agricultural lowland 
at a convenient meeting place for lake and land routes, is the chief 
railroad center of the United States.” 

*The fact that Mexico is not a leading world power is explained 
in part by scarcity of mineral resources there.” 

“A major factor in the growth of London has been its location 
on one of the large producing coal fields of England.” 

Group III. Associating correctly with given places, countries, 
or groups of countries specific facts or relationships. 

Place or region 


Item given Type of fact or relationship to be evaluated concerning it 

47 northeastern 

United States type of surface and transporution net 

^ Chicago location; nature of tribuMry area 
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Place or regioii j • • 

Item given Type of fact or relationahip to be evaluated concerning it 


atate in the n ■ j c 

United State* location; features in northeastern United States related to 
mining in this state 

50 New Jersey . . soil, drainage, density of population 

65 Mexico world power in relation to mineral resources 

66 Mexico world power in relation to farm, forest, and grazing resources 

67 Mexico world power in relation to climate 

68 Mexico world power in relation to human characteristics 

69 Brazil distribution of people 

70 Brazil mineral resources; forest resources; importance of agriculture 

71 Brazil surface and soil of eastern and southern parts 

72 Brazil outside communication in relation to nature of coast 

73 Brazil nature and distribution of agriculture in relation to soil and 

climate 

80 London growth as related to location and nature of river 

81 London growth as related to location with regard to coal fields 

82 London growth as related to natural route focus and government work 

83 London growth as related to location with regard to sea routes 

89 southwestern 

Asia sparsity of population as related to location with regard to 

sea routes 

90 Bouthwestem 

Asia sparsity of population as related to location with regard to 

land routes 

91 southwestern 

Asia sparsity of population as related to climate 

Sll^^^rauntries of 

i^^ Murope colonizing activities 

/f^Swogth America colonizing activities 

Sooth America political status (independence vs. dependency) 

9S countries of 

Africa political sratu s 

96 ' edoniidnf ' 

”*“”**^* • • • colonizing activities in relation to size of coloniMr 

97 colons^ 

epuntwes. . ■ . colonizing activities in relation to density of popula tion 
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Place or region 
Item given 

Type of fact or relationship to be evaluated concerning it 

98 

colonizing 
countries .... 

colonizing activities in relation to latitude 

99 

colonizing 
countries .... 

colonizing activities in relation to home resources in propor- 
tion to size of country 

100 colonizing 
countries. . . . 

colonizing activities in relation to home advantages for trade 
and manufacturing 


A fourth group of items tests for the ability to associate correctly 
with given facts (read from pictures, descriptions, or graphs) other 
facts, not given, which are typically true of places where the given 
conditions are found. For example: 

‘‘People who live in places like that shown in Figure 1 [picture] 
prepare for Idng winters.’' 

“In the state referred to in paragraph VIII [a description] can- 
ning factories, saw mills, and flour mills are probably more numer- 
ous than iron, steel, and textile mills.” This involves an under- 
standing of relationships existing between these types of manufac- 
turing establishments (not mentioned in the description) and the 
production of specific types of raw materials mentioned in the 
paragraph. 

“Population probably is evenly distributed throughout the 
area [represented by accompanying graphs].” This involves the 
ability to read from the graphs that rainfall in the region 
sented is scant, that more than ninety-five per cent of the 
desert, but that thousands of acres are cultivated; it involvd^l^ 
the ability to associate with those facts uneven distribution of 
population, about which the graphs gave no direct information. 

The items included in Gi^oup IV are listed below, together with 
the types of sources from which the given facts were to be gleaned 
and the types of facts (not given in these sources) to be associated 
with those given. 
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Group IV. Associating correctly with given facts various other 
facts and relationships normally related to them. 

Sour« of fxcts 

Item given Types of associations involved 

2 picture use of materials for food, 

clothing and shelter vegetation indicative of low 

latitude; of middle latitude 

5 picture...... length of seasons vegetation indicative of low 

latitude 

6 picture amount of rainfall vegetation indicative of low 

latitude 

7 picture length of days vegetation indicative of low 

latitude 

8 picture position of noon sun vegetation indicative of low 

latitude 

9 picture length of seasons vegetation indicative of low 

latitude; of middle latitude 


23 

description. . . 

amount of rainfall; irrigation. 

nature of houses, food, work 

24 

description . . . 

day and night temperatures; 
length of seasons 

nature of houses, food, work 

27 

description. . . 

aridity 

nature of houses, food, work 

28 

description, . . 

length of growing season 

nature of crops grown 

29 

description. . . 

time of occurrence of season . 

harvesting activities 

37 

description. . . 

length of growing season; 
amount of summer rainfall. 

nature of crops grown 

43 

description. . . 

kinds of manufacturing work. 

agricultural, mining, and log- 




ging activities 

51 

description. . . 

seasonal distribution of rain- 
fall 

nature of crops grown 

graph 

distribution of population.. . . 

climate; land utilization 

n 



importance of agricultural 




work 

nature of trade 

■ 

>f 




One of the hundred items in the test cannot be classified as be- 
liHIgtng to any of the groups designated. It is item 75; “Rainfall 
i^ounts to kbout 10 inches a year.” It refers to a rainfall graph 
Wd pi^^y for the ability to read the graph. It was included 
for dii^iliosttc purposes. 


1260 ] 







ANALYSIS OF RESULTS 


It should be clear from this analysis of the test that much fact 
knowledge was called for; it should also be clear that the test 
called for much a given description, some of the facts 

may fit a certain region, but not all of them. One must weigh the 
group carefully to reach a correct decision. To react correctly and 
consistently to many of the items, one must have knowledge of 
interrelated human and natural facts which characterize the re- 
gions or places in question and must be able to see relationships 
between outstanding types of facts. 

It seems clear to the committee that unless knowledge and 
abilities of the types represented in Parts I and II do result from 
the study of geography in elementary and secondary schools, 
much of the opportunity of contributing to an ability to pursue 
social studies effectively is being lost. 

Analysis of Results 

The administration of the tests and the compilation of data 
secured through their use were not under the auspices of the 
committee. The results were reported to the committee in the 
form of item profiles. In the case of each item in the tests there 
was recorded, for each grade or level in which that exercise was 
tried, the percentage of the pupils tested who reacted correctly. 
The college freshman year is designated as grade 13, and the 
Stanford and Princeton freshman groups were assigned to Grade 
13 +. These data were examined with a view to discovering what 
light they shed on the answer to five major questions, namely: 

1. What evidence do the data furnish concerning the reliability 
or dependability of the exercises as instruments for taking an 
inventory of the stock of fundamental geographic achievements of 
groups of pupils? 

2. In so far as the exercises are dependable, what do the respitin'* 
show about the attainments of the groups tested ? 

3. What do the results reveal concerning progress at various 
levels toward the goals involved ? 

4. What light do they shed upon weaknesses in the training of 
pupils in geography and upon remedial planning ? 
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5. What light do they shed upon needs to be met in planning 
for further testing? 

In answering these questions, results of the use of Part 11 were 
analyzed first. Consideration of the dependability of the items 
involved was based on the following two understandings: 

1. A test item is dependable if it has validity, that is, if it tests 
for the specific ability for which it was designed to test, and if it 
has reliability, that is, if reactions to it are consistent with the 
ability known to be possessed by those who react to it. 

2. Reliability of an item can be established only in terms, then, 
of two factors, namely: (1) reactions of individuals or groups to that 
item and (2) other evidences of possession of the ability involved, 
or of lack of it, on the part of the individuals or groups reacting. 

All the data furnished to the committee had to do with the reac- 
tions of individuals and groups to items in the test itself. Though 
four forms of the map-reading test had been supplied by the com- 
mittee for the purpose of securing several evidences concerning 
the abilities involved, these forms were not administered to the same 
groups. Reliability obviously cannot be established merely in terms 
of the performance of different groups with corresponding items. 
The data furnished, then, are inadequate for the purpose of deter- 
mining dependability. However, certain general facts about the 
abilities of the groups tested are known, and the nature of the test 
itself is such that these data, when considered in the light of what is 
known about the abilities of the performers^ throw some light on the 
matter. 

As to validity, the nature of these questions is such that it is 
di^ult to conceive of their testing anything other than the specific 
a^Hty they are designed to test. The language is exceedingly 
simple and the statements are perfectly definite. Take, for example, 
Qtt statement, “The latitude of J in Figure 9 is 60° N.” If one can 
tcad latitude accurately, he can tell from the map that this state- 
let is true and can r^act correctly. If he does not have that 
he may react correctly by chance. If he reacts correctly to 

^ ot Vi 

irsouti .maicfeB correctly, the chance element is outweighed. ^^^Rt 
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ability could consistently correct performance in this case possibly 
evidence other than the ability to read latitude from maps of the 
type used ? Logic prompts one to answer “None.” 

“In Figure 12, arrow R is pointing due south.” What ability 
other than the ability to read directions from a map drawn, as 
is “Figure 12,” on a polar projection possibly could be evidenced 
by consistently correct reactions to this statement? What could 
consistently incorrect reactions evidence except inability to read 
directions from such a map ? 

Such considerations, though subjective, should have weight in 
evaluating testing material. 

To gain further evidence concerning validity, the following 
experiment was conducted. A group of twenty-five adult students 
(college seniors and graduates) who were thought to possess the 
map-reading abilities involved were given several informal tests 
to secure evidence to that effect. When satisfactory evidence was 
in hand, these students were given the tests involved in this dis- 
cussion and were asked not only to react to each statement, but 
also to state the specific ability for which each item tested. Twenty- 
one of them made perfect scores (50), and four made scores of 48 
or 49, having erred either in their count of contour lines in deter- 
mining elevation or in calculating range in longitude or in both. 
In no case did they fail to recognize the specific ability for which 
each given exercise called. 

These exercises were given afterward to a group of fourteen 
students whose map-reading ability was an unknown quantity. 
Each individual of the group was tested separately on every phase 
of ability about which his test performance left doubt. It was 
established by this procedure that six per cent of the test errors 
were due to carelessness, and that 94 per cent of them were due to 
lack of requisite ability. These students, like the others, listed 
correctly the abilities for which the test called, even in the cases 
upon which they failed. They recognized what each exercise called 
for, whether or not they could respond correctly. The results of 
this experimentation supported the subjective conclusion concern- 
ing validity. 
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In the construction of the map test, the committee had provided 
carefully for another means of checking on dependability. The 
ability to recognize and name semi-pictorial symbols is one which 
practically every normal individual acquires relatively early in his 
experience with maps and one which he tends to retain. One recog- 
^•nizes some of these symbols almost as readily as he recognizes 
features in picture, because of their semi-pictorial character. He 
may recognize some of them without any technical training. To 
recognize certain combinations of these symbols which represent 
straits, peninsulas, and the like, he needs to learn what straits are, 
what peninsulas are, and so on. These terms are introduced rela- 
tively early in most, if not all, courses in elementary geography. 
This phase of ability constituted, in essence then, a known quan- 
tity. Care was taken to devote about a tenth of the test to items 
of this type calling for abilities which one scarecely could fail to 
have developed, whatever his geographic experiences had been. 
Consistently correct reactions to these items would show that test 
results relating to them were in accord with known facts concerning 
the ability of the groups tested. These items in Form I of Part II 
were 1, 2, 3, 4, and 5. The performance of all groups on these 
items was high. The lowest percentage of correct reactions by any 
group of pupils to any one of them was 84, which was the percent- 
age of correct reactions that sixth grade pupils made to item three. 
This percentage was high for sixth grade pupils, for it was reached 
or exceeded in the case of only seven items out of the fifty involved. 
The lowest percentage in the college freshmen group was 94. 
Results from the use of this group of items, then, were in harmony 
with what were essentially “known abilities” on the part of those 
tested. 

Some definite knowledge also is possessed concerning the ability 
of adults to use map networks effectively in determining the rela- 
tive size of two areas. Dr. Helen M. Strong of the United States 
Department of Commerce experienced so much difficulty from this 
toource on the part of business men seeking information that she 
hroiftglsNt ab^ut the publication by the department of an equal area 
world map for their use. It was designed to help in overcoming the 
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inability to recognize distortion of areas on the mercator or any 
cylindrical projection. Item 38 tested for this ability to recognize 
the difference in size of ^^reas (mapped on a cylindrical projection) 
that extended through the same number of degrees of latitude and 
longitude but were in different latitudes. Consistently low per- 
formance on this item would be in accord with knowledge concern- 
ing probable lack of ability of those tested. The percentage of 
correct responses on the item varied from 0 to 33. To no other 
item was there so low a percentage of correct responses. 

Mr. F. £. Lord, director of the training of rural teachers at the 
State Teachers College, Ypsilanti, Michigan, had carried on a 
scientific investigation of the ability of seventh and eighth grade 
children to read latitude and longitude. His findings afford clear 
indication of the fact that pupils who possess these phases of ability 
are fewer than those who possess ability to read semi-pictorial 
symbols. In the light of his investigation, the percentages of correct 
reactions to items 22, 23, 25, 26, and 30 should be less than those 
to items 1, 2, 3, 4, and 5. The percentages of correct reactions by 
the sixth grade pupils to the questions ranged from 50 on item 30 
to 69 on item 26, as against 84 or more on the semi-pictorial items. 
The percentages of correct reactions by college freshmen to these 
items ranged from 73, on each of items 22, 23, and 25, to 83 on 
item 30. The performance of these groups is typical of the per- 
formance of all groups tested. The results, then, are in harmony 
with those of Mr. Lord’s investigation. 

Other examples of similar character might be added, but these 
suffice to illustrate the point that in so far as the knowledge of the 
second factor had been obtained, performance on the tests was in 
harmony with what was known about the possession of the ahilities 
for which the exercises test. Since all the |;est items were similar in 
form, were expressed in language as simple as the requisite technical 
terms permitted, and were different only in the types of ability for 
which they called, it seems altogether probable that (with ex«p- 
tions to be noted later) all the items were as dependable as those 
for which data were available for comparisons of the type just 
indicated. 
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Comparison of the data from Princeton and Stanford freshmen 
with data from other freshmen lends strength to this probability. 
The former group was separated from the latter because its mem- 
bers were known to have had somewhat more traihing in geography. 
In the case of two items, the percentages of correct performance in 
the higher and tower ability groups were the same. In forty-six 
cases, all other cases save two, the percentages of correct reactions 
by the group with more training were from 2 to 29 per cent higher 
than those by the other group. 

Another procedure followed in seeking light upon the depend- 
ability of the exercises was the comparison of percentages of cor- 
rect performance on various items by the various groups of college 
freshmen tested. A record of the reactions of the 1433 individuals 
comprising these groups was secured from the Commission. Except 
for the Princeton and Stanford groups, no data were furnished 
concerning the general abilities of these groups, the amount of 
geographic training they had had, and the character of that train- 
ing. The amount and character of geographic training are known 
to vary greatly in different school systems. In some, there is no 
geographic training beyond that received in grade six; in others, 
none beyond grade seven; in still others, none beyond grade eight, 
ted so on. The fact that these people had reached college level is 
the only indication furnished of their ability, and obviously, under 
the conditions noted, their geographic experiences doubtless had 
varied widely. No very close correlation in their products could 
be expected under these circumstances, but the accompanying 
graphs were constructed with a view to seeing what consistency, 
if any, there was in their performances. A general similarity is at 
once discernible. Although there was considerable range, as was 
to be expected, in the per^ntages on a given item, it is to be noted : 
l3iat all groups performed best with the first third of the test, and 
JeXM: wdl with the last third of it; that high points in the various 
occur in connection with the same items; and that in con- 
itectiai|,^with another gi^oup of items low points occur in each. 
T^je consistency of performance is striking and lends sup- 

|>ort tb a bdief in the dependability of the test. 
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A procedure commonly followed in determining the reliability 
of a given test is that of finding the coefficient of correlation be* 
tween the scores of one hundred random samples of individuals on 
chance halves of the test. If the items in a test are all concerned 
with the same type of ability or with different phases of that 
ability which are of approximately the same difficulty and are 
functions one of another or of the same learning experiences, there 
should be marked positive correlation between these scores. If one 
thinks logically about such a procedure, however, he realizes that 
if a test deals with a wide range of abilities, some of which are 
known to be possessed by many, and others of which are known to 
be possessed by few — abilities which differ much as to the difficulty 
of developing them and as to the specific experiences necessary in 
so doing — little correlation should be expected between chance 
halves of the test as a whole. A low coefficient in this case would 
not mean unreliability. It is known, for example, that little corre- 
lation exists between one’s ability to identify a strait or river or 
to read elevation from a color-contour map, and one’s ability to 
read a polar projection, to read altitude from contours, or to trans- 
late distance in degrees of longitude into distance in miles at 
various latitudes. The only thing in common in the various abili- 
ties is that they relate to maps. They involve, however, the interpre- 
tation of entirely different symbols. It is known that nearly all 
people have had experiences which, however meager, lead to an 
ability to interpret some simple symbols. It also is known that the 
experiences of a vast majority of people have not been such as to 
lead to the ability to translate various other types of symbols. 
One logically could no more expect high correlation between per- 
formances on halves of a test which do not cover the same types 
of map-reading ability than he could expect it between two tests, 
one in American history and one in English history, if he knew 
that those tested had had experience with the one, and had not 
had experience with the other. Both tests are in history, but 
knowledge of American history does not breed knowledge of 
English history. Speaking mathematically, one is not a function 
of the other or of the same group of experiences. Two distity^ 
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sets of experiences are needed to acquire knowledge in both fields 
of history. The same is true with reading semi-pictorial map 
symbols and other types of map symbols. 

*Tn order to apply this rule of chance halves in the only way 
which is sound, then, the test had to be divided into groups of 
questions between which some degree of correlation might be 
thought, with reason, to exist. Then one could select chance halves 
of each of these groups. The halves of the test as a whole would 
thus consist of the chance halves from each group. Several exer- 
cises, for example, deal with getting the simpler types of ideas to 
be gained from a map network. Several others deal with getting 
the more complex types of ideas from a map network. Others deal 
with mere recognition of features from semi-pictorial symbols. 
Another group deals with reading various types of ideas from such 
symbols, and so on. From each of these groups, chance halves 
WMe taken. Even these halves were not such that an exceedingly 
high correlation logically could be expected. One may well read 
latitude without having the ability to relate latitude and crops, 
and these were grouped together. One may read latitude without 
being able to read longitude. One may be able to translate degrees 
of latitude into approximate distance in miles without being able 
to translate degrees of longitude similarly, and so on. The only 
halves of a test between which perfect correlation could be expected 
would be halves in which abilities were paired exactly. 

There were not enough items in the test to permit such exact 
matching. Even with the rough matching possible, however, the 
coefficient of correlation between the scores of one hundred fresh- 
men, selected at random from the 968, on the halves of the test 
consisting of chance halves from each group is .53. In view of the 
nature of the grouping, a correlation of .53 is satisfactory as evi- 
dence of reliability:. 

It is unfortunate that the plan of the committee for establishing 
reliabSity^thirough the use of two or more carefully matched forms 
the same groups was not effected. Since there was consider- 
ll^ll lranige in the performance of different groups at freshman level 
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with a given form, the performance of different groups with dif- 
ferent forms gives an unsatisfactory basis for comparison. 

It is significant, however, that the performance of different 
groups in given items in Form I, True-False, parallels rather closely 
that of different groups on qther forms of those items. For example, 
the percentage of three college freshman groups reacting correctly 
to item 1, Form I, True-False, were 94, 93 and 95. The percentage 
of correct reaction on the part of all freshmen to this item was 94. 
This represents a group of 968. Form II, True-False, was tried on 
only 48 freshmen. Obviously, percentages computed on such dif- 
ferent bases as 968 and 48 are of little value for comparison. The 
other forms, tried with 336 and 181 individuals afford better bases. 
The range of performance of the three groups on the same item 
(No. 1, Part II, T. F.) was 93-95. The range of performance of 
three other groups on that item and corresponding ones in different 
forms was 94-99. In the case of this particular exercise, the multiple 
choice form gives one somewhat more guidance in answering than 
does the true-false form. This fact, and the differences in the num- 
bers of cases involved, doubtless account for the somewhat wider 
range on the different forms. In many other cases, the performance 
of three groups on the same question coincides closely with that of 
three other groups with three forms of the same question, as the 
following table shows. 


Item in 

Form I 
True-False 

Range of performance of 
three groups on the 
same item 

Range of performance of 
three ocher groups on 
different forms of a 
given item 

2 

95-96 

96-97 

3 

94-95 

91-95 

5 

94-94 

93-95 

10 

70-87 

68-86 

11 

94-97 

95-97 

13 

93-98 

94r-98 

24 

60-68 

63-65 

25 

81-85 

81-82 

27 

62-77 

65-77 

36 

44-72 

49-65 

44 

44-72 

39-73 

45 

28-37 

27-41 
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In some cases^ the range of performance on the same item by 
different groups was under than the range of performance by dif- 
ferent groups on different forms of that item. For example; 


17 

84-93 

90-95 

21 

76-85 

78-79 

23 

69-80 

70-73 

28 

41-57 

45-51 

30 

79-89 

80-83 

32 

64-72 

59-63 

37 

43-56 

51-62 


The consistency of performance of six different, large groups 
indicated by these tables certainly is not a matter of chance. The 
fact that reactions to different forms show about the same results 
as reactions of different groups to the same form points strongly 
to the probability that, if the various forms had been given to the 
same group, there miu}d have been very high correlation between 
the reactions of any given group on the several forms. If this be 
the case, then obviously the exercises are dependable and the 
differences in the performance of groups could be identified unques- 
tionably with differences in their geographic abilities. Even in 
cases where the range was greater on the different forms than that 
on reactions of different groups to the same form, there is a marked 
coincidence of performance. For example: 


57- 74- 
77-93 

58- 83 
74-85 
58-75 
69-75 
50-65 
50-65 

6-22 

40-^5 

37-56 

35-75 


6 

6J-70 

8 

85-92 

9 

51-73 

14 

82-87 

15 

71-77 

19 

73-77 

20 

64-66 

34 

62-67 

38 

11-14 

39 

54-71 

41 

44-57 

48 

47-75 
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The findings from the foregoing considerations concerning the 
reliability of the test items in Part II may be summarized as 
follows: 

1. The data furnished are inadequate as a basis for determin- 
ing the true coefficient of reliability. 

2. They serve, however, when handled scientifically, to estab- 
lish the fact that the items have much reliability, since .53 
is, under the conditions noted, a satisfactory coefficient 
of correlation. 

3. Further evidences that the test has much reliability are: 

(1) Close correlation between performance on it and defi- 
nitely known abilities of two selected college groups 
tested. 

(2) Qose correlation between performance on it and abili- 
ties or lack of abilities known from investigations on 
the part of others. 

(3) Performance by a “higher ability” college freshman 

? [roup that proved consistently better than the per- 
ormance of a “lower ability” group. 

(4) A marked tendency toward consistency of performance 
on the part of six college groups, both on the same item 
and on different forms of that item. 

There is no conclusive evidence of unreliability save in the cases 
of two items and possibly a third, to be noted later. 

Statistics are of value to science only when used scientifically. 
When two factors are involved, as in the case of reliability, it is 
unscientific to base a conclusion on data dealing with only one of 
those factors. A valid conclusion can be reached only through 
careful consideration of both. In using chance halves of a test, 
selected without recognition of the known fact that abilities in the 
two halves are as different as abilities in two different fields, and 
in basing conclusions concerning reliability of the test on correla- 
tion between performance on those two halves, one is using only 
one factor and ignoring the other. One can not question the fact 
that the coefficient of correlation between such chance halves is 
low. Scientific thinking leads one to challenge, however, the state- 
ment in the report of the use of the test that this coefficient is a 
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coefficient of reliability. It is only a coefficient of correlation be- 
tween two tests (considering each chance half as a test) on two 
different and only slightly correlated helds. The coefficient .53 
undoubtedly is nearer the true coefficient of reliability than the 
coefficient .055 submitted to the committee in the report men- 
tioned. It seems highly probable that, if complete data were in 
hand, a coefficient of reliability even higher than .53 would be 
established conclusively. 

In connection with Part I, the same procedure was followed with 
similar results. The accompanying graphs^ indicate the perform- 
ance of various college freshman groups with this part. The coeffi- 
cient of correlation between scores of one hundred freshmen, 
selected at random, on halves of the test consisting of chance 
halves of correlatable groups of items is .67. 

Mr. Floyd E. Masten, of the University of Chicago, who ren- 
dered valuable assistance to the committee in connection with the 
statistical work involved in the report, points out that this coeffi- 
cient does not mean necessarily that Part I is more reliable than 
Part II, because it was based on a consideration of twice as many 
items. Since there are one hundred items in Part I and only fifty 
in Part II, .halves of Part I were larger than those of Part II. 
Obviously, the coefficients .53 and .67 are not the true coefficients 
of reliability, for in neither case was the scope of abilities covered 
in the two halves of the test identical, but in all probability the 
true reliability coefficients, which can be determined only after ade- 
quate data concerning both factors involved are in hand, is higher 
rather than lower than these coefficients indicate. The foregoing 
considerations lead to the conclusion that the test is reasonably 
rdiable as a group test for the purpose for which it was designed, 
namely, to determine what fundamental geographic abilities have 
been attained by students at various levels of school work. 

ISince in a given form an individual made only one reaction in 
the case of most of the abilities involved, it obviously is not 
possible with the use of k single form to tell which of the correct 
reactions on the part of that individual were a matter of knowledge 
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and which a matter of chance. If, however, all fou^ forms are Qsed 
with the same individuals, Part^Il is useful also in diagnosing 
individual cases. 

Attention was turned next to the second question, namely, “In 
so far as the exercises are de^ndable, vdiat do the results show 
about the attainments of the groups tested?” In order to have a 
convenient standard of comparison, it was decided to accept as 
satisfactory evidence of TOssession of a given ability on the part 
of a group the fact that 75 per cent or more of the group made 
correct reactions to the exercise which called for that ability. This 
choice, of course, was purely arbitrary. One might have chosen 
70%, 80%, or some other at that general level. The only requisite 
was that the percentage chosen be high enough so it could not 
represent chance performance to any significant degree. The use 
of such a percentage would at least serve to separate abilities 
possessed by many from those possessed by comparatively few 
individuals. 

The findings with regard to map-reading abilities were as follows: 

Seventy-five per cent or more of the 225 sixth grade pupils 
tested reacted correctly to items 1, 2, 3, 4, 5, 11, 13 and 17 in Form 
I, Part II, True-False. No other form was tried with sixth grade 
classes. The ^^ilities with which these items are concerned are the 
foillowihg: 

Use of semi-pictoriai symbols: 

Ability to identify » 


1. land and Sea 

2, railroads..., 


* 3j|: maitt 

citie%and mountainous land 

^ Sk die j^dierif versus the northern part of a giveiTregion 

versus other types of toad (with aid of legend). . . 

Ip^iedltir iq^^eynibols: 

^1^ " MMkyjSO ratify land Ing^t above sea levd in a gi^Mpf^n. 

% ' 


(87%) 

i^%) 

(84)% 

(8S%> 

(89%) 

(85%) 

(85%) 

( 77 %) 
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-< 4f 

It IS to be noted that these eight abilities can be acquired more 
easily than any others involved in the test. It is in harmony with 
known facts about courses in fourth, fifth and sixth grade geography 
that pupils should have acquired them by the time they are in 
the sixth grade. As a matter of fact, the first six of them probably 
were acquired in the course of the fourth grade work and common 
intelligence makes it possible for one to tell with the aid of the 
legend which land in “Figure 4’* is highest, even without previous 
experience with such a map. 

Other abilities which were evidenced less conclusively, as the 
percentages in the parentheses indicate, were those involved in 
items 7, 8, 10, 12, 14, 15, 19, 21, 23, 25, 26 and 27, namely: 

Use of semi-pictorial symbols: 

Ability to get ideas of: 

7. relative location of river and railroad 

8. comparative distance without the use of a scale of miles 

10. relationship between gridiron street pattern and topography 

12. distance and highway marking as shown on auto road map 

Use of color spot symbols: 

14. Ability to read and compare facts about density of population and 

amount of rainfall 

15. Ability to see probable relationship between facts read about crops 

and rainfall 

Use of straight-line network: 

21, Ability to read direction from the equator 

23. Ability to use facts read about latitude in interpreting type of vege- 
tation 

Use of curved-line network: 

19. Ability to read direction 

25. Ability to read latitude in terms of degrees 

215. Ability to read latitude in terms of comparison with^distance from 

equator to pole 

27. Ability to use facu read about latitude in interpreting length, of 
growing season. . . .* 

The perfoimimce of the sixth grade with the test was alniost 
exactly wha| to be expected in the light t>f known conditions. 
It is to be notajlplhat each of the second group of abilities, involving 
the use of semii^ctorial symbols is of a higher order than any cd* 
those in the first group. In items 7, 8 and 10, companso^ wietf 
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( 71 %) 

(66%) 

( 61 %) 


( 67 %) 

( 61 %) 

( 65 %) 

(6f%) 


( 62 %) 

( 60 %) 

( 61 %) 

( 69 %) 



TESTS AND MEASUREMENTS 


involved, and in item 12 one has to deal with two types of figures 
on the map — those indicating mileage and those indicating high- 
way numbering. Comparison also was involved in the exercises 
using color spot symbols, whereas in the one ability evidenced 
in the use of these symbols in the first group, no comparison was 
called for. It obviously is more difficult to read latitude and to 
relate it to vegetation and growing seasons than it is to read north 
on a straight line network. Nor is it surprising, in view of the 
maps in “Figures 8 and 9,“ that latitude was read correctly from 
the curved network (Item 25) in more cases than from the straight 
network (Item 22). Three facts make the lt|^r a more difficult 
exercise. J in “Figure 9“ is exactly on a pffkllel at the end of 
which is the number 60. In contrast, E in “Figure 8” is not on a 
parallel, and again the matter of comparison enters in. Secondly, 
all places in “Figure 9“ are in north latitude, whereas all places in 
“Figure 8” are in south latitude except E. Thirdly, the converg- 
ence of meridians at the north pole in “Figure 9” is a clear indica- 
tion of “northness” with regard to the equator. It at once suggests 
the north pole. In contrast, the only basis furnished in “Figure 
8” is the numbering of the parallels and one must identify 0*^ 
with the equator and make comparisons based on that identihca- 
tkm in order to realize that E is in north latitude. The point to be 
msdi is^liat the sixth grade pupils tested could read latitude, 
g^ven the simplest means of so doing, but could not read it if these 
aids were not at hand. They had acquired only ability to use the 
more obvious sjgns of latitude. 

In the case of the 29 items to which less than 60% of the sixth 
grade reacted correctly, the probability is that the abilities called 
for are not possessed generally by the group. 

In the' case of one item there is clear evidence that a 62% 
oorrect reaction does not evidence ability. This fact points to a 
weakness in die test. Item 42 was designed to call for the ability 
to Dead and compare range in temperature between two sets of 
pieces. Kno^edge of experiences in elementary^|eography leads 
erne to «pect only chance correct answers to this item on the part 
ibf sixth grade pupils. This was the case with the other three exer- 
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cises dealing with the ability to interpret isotherms^ a fact which 
proved that this percentage meant either unusually fortunate 
guessing or some weakness in the exercise. Examination of the 
map in "Figure 13” on which the question was based shows that 
the number 54 is near Gj*the number 56 near H, and the number 
64 near E. Though the figure near H does not represent tempera- 
ture at H, the result gained from reading these figures instead of 
reading the isotherms gives the same result as would correct inter- 
pretation of the isotherms. Apparently, the children used these 
numbers instead of the isotherms. This possibility could have been 
obvitited by omitti^ the numbers on the map on all isotherms 
except 64, 62 and 

Significant facts about the reactions of college freshmen (exclud- 
ing the Stanford and Princeton groups) are the following: 

1. Eighty-five per cent or more of the college freshmen reacted 
correctly to all eight items in the first sixth-grade list. 

2. Eighty per cent or more of them also reacted correctly to 
items 8, 10, 14, 25, and 26 in the second sixth-grade list; seventy-five 
per cent or more of them to items 7, 15, 19, and 21 in that list; and, 
in addition, to items 30, 35, 48, and 49. 

3. Whereas only eight items were handled correctly by three- 
fourths or more of the sixth-grade pupils, then, twenty-one items 
were handled correctly by three-fourths or more of the college 
freshmen. These twenty-one, moreover, included the eight. 

4. Between sixty and seventy-five per cent of the freshmen 
reacted correctly to items 6, 12, 16, 20, 22, 23, 24, 27, 29, 32, 34, 36, 
39, and 44. 

5. Whereas twenty items, then, were reacted to correctly by 
sixty per cent or more of the sixth-grade pupils tested, thirty-five 
items were treated correctly by sixty per cent or more of the collie 
freshmen, and these included all the twenty items. The difference/ 
between the forty per cent of the test handled fairly satisfactmily 
by the sixth-grade pupils and the seventy per cent of it handled 
equally well or better by college freshmen indicates clearly that 
gain was made in map-reading ability between the sixth and thir- 
teenth school years. 
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6. Greater average ability on the part of the freshmen as com- 
pared with that on the part of the sixth-grade pupils also is evi- 
denced by the fact that freshmen percentages were in no cases as 
low as percentages on corresponding items in the sixth grade; and 
that in the cases of fourteen of the twenty items in the sixth-grade 
lists, percentages were from nine to twenty-two per cent higher. 

7. Items to which less than sixty per cent of the college freshmen 
reacted correctly were those numbered 9, 18, 28, 31, 33, 37, 38, 
40, 41, 43, 45, 46, 47, and 50. (Item 42 was not included in these 
comparisons because of the objection noted to it in the earlier 
discussion.) Item 18 deals with east-west direction on a curved 
line projection; item 28, with the translation of distance in degrees 
of latitude to distance in miles; item 31, with longitude in degrees 
as compared with distance between the prime meridian and the 
180th meridian; item 33, with the comparison of areas of regions 
mapped on different projections; item 38, with the comparison of 
areas of regions in different latitudes mapped on a straight-line 
network; item 37, with the relation of the continent pattern to lati- 
tude and longitude; item 40, with recognition of meridians on a 
polar network; items 41-43, with reading isotherms; items 45-47, 
with reading isobars; and item 50, with the identification of topo- 
graphic profiles. These obviously are the more difficult phases of 
map-reading ability represented in the test. If any gaps are left 
in the ability to read maps in common use as a result of elementary 
and secondary training, those evidenced by incorrect reactions to 
the items just listed are the ones confidently to be expected. This 
expectation is based on (1) the relative difficulty of acquiring the 
abilities in question and (2) established facts concerning current 
teaching procedures. 

8. With the exception of item 9, then, performance on the test 
coincides with the prediction of results on the part of the committee 
— a prediction based on definite knowledge of the two factors noted 
at the end of the preceding paragraph. 

9. Item 9 calls for the use of a scale of miles. Evidence of lack 
of the ability involved was surprising. It may mean that, though 
the tool is introduced at early levels, practice in its use is insufficient 
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to establish the hahit of using it accurately. Perhaps many of those 
tested merely measured by eye comparison and did not trouble to 
measure the distance in the proper way. This possible explanation 
is in accord with the factnthat performance was better on the cor- 
responding item in the other forms. There is the third possibility 
that the term “about” was interpreted more liberally by some 
than by others. If so, the weakness is in the statement itself. 
The matter cannot be settled without further experimentation. 

10. Summarizing, the results of the use of the map-reading tests 
give a clear picture of the probable gaps in map-reading ability 
which are left at the end of secondary school training, as it is com- 
monly administered, and give a very definite notion of the type 
of advancement made in reading maps in grades six through 
twelve. The tests were, therefore, highly satisfactory for this one 
of the purposes for which they were designed. 

The discussion of the value of Part I in answering the second 
major question, page 261, can be handled more briefly in connec- 
tion with the discussion of the third question, namely: “What do 
the results [of the use of the tests] reveal concerning progress at 
various levels toward the goals involved?” Facts about known 
conditions that must be considered carefully as constituting one 
type of factor requisite to sound conclusions in the matter are the 
following: 

1. There is no uniformity in the experience provided in so-called 
geography courses beyond the sixth grade, and little uniformity in 
those provided earlier. 

2. Some of the abilities involved are those which tend to be 
developed, as one progresses toward maturity, through chance 
experiences, or through experiences in other fields. Any intelligent 
twelfth grade child who has had an effective course in European 
history, for example, would react correctly to more items in these 
tests than would a child of equal intelligence who has had other 
experiences like those of the first child but has had no such history 
course. 
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3. Reading habits of individuals vary greatly, and the more 
widely an individual reads, the more he tends to glean information 
which contributes to geographic abilities. 

4. Many abilities tend to atrophy with disuse. Assumption that 
any given ability is possessed by a greater number of people in 
each successively higher grade ignores the significant factor of the 
degree to which the ability has been used in the interim. 

5. Groups in the same grade vary greatly in general intelligence. 

It also is to be noted that: 

1. The percentages in the item profiles submitted are computed 
on bases ranging from 3 to 968. To use in an item profile per- 
centages based on 3, 13, 26, 74, and 92 cases, in comparison with 
those based on several hundred cases, tends to give erroneous 
impressions. 

2. If what a given group does in the sixth-grade is compared 
with what that group does in the seventh grade, the eighth grade, 
and higher grades, or with what is done at higher levels by another 
group known to have had the same learning experiences in grades 
below the sixth and to possess the same intelligence, the factors of 
difference in experience and intelligence would not need to be con- 
sidered in interpreting a profile. The profiles submitted to the 
committee, however, are not of this type. They compare what one 
group did in the sixth grade with what a different group did in the 
seventh grade, a third group in the eighth, and so on. 

By reason of the facts listed above about variations in groups, a 
variable which is exceedingly erratic must be taken into consid- 
eration in any sound interpretation of these profiles. In the report 
of the use of the tests to the committee, the profiles are interpreted 
as if they were profiles of the first type described; the second 
variable involved is ignored. 

3. If profiles of the second type noted “fail to show appreciable 
grade-to-grade improvement” and “a large number of mean scores 
of items” are “of the same order of magnitude as chance scores,” 
the items in question are classed in the aforementioned report as 
“poorer” items. It is also stated that such profiles indicate that 
the items “in question will not function as a measuring device for 
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the grades in question.” As well might it be said that a yard stick 
is not a good device for measuring the height of children (a variable 
which tends to increase from year to year) if one happens to use it 
on a group of children who, for some reason such as under-nourish- 
ment or thyroid abnormality, had not made the growth to be 
expected under good conditions. 

Such reasoning concerning item profiles assumes that promotion 
from one grade to another inevitably is accompanied by appreciable 
improvement in the abilities in question, whatever the intelligence 
of the groups involved and whatever the nature of their learning 
experiences. This assumption is unquestionably false, and reason- 
ing based on it therefore must result in unsound conclusions. 

Any item is a good “yard stick” for measuring progress if reac- 
tions to it are consistently in accord with known abilities on the 
part of those who react. In the report, the unquestionably reliable 
item, "The distance from J to M is about 1400 miles” is marked 
“poor” as a measuring device because 47 per cent of the sixth- 
grade pupils tested reacted to it correctly, 52 per cent of the 
seventh-grade pupils, 45 per cent of the eighth grade pupils, and 
so on. What does nearness to the chance score mean? It means 
inability on the part of the majority of people in all three groups. 
One who knows does not resort to guessing. The yard stick shows, 
in other words, that children of all three groups are not as tall as 
they should be under conditions of proper nourishment. The school 
physician who made such a finding would not question his yard- 
stick. He would try to find out what facts about the diet of these 
groups or about other functioning conditions of growth accounted 
for the retarded growth of individuals in the groups involved. 
Knowing beyond question that one who has the ability to read 
latitude and to translate degrees of latitude into approximate 
mileage reacts correctly with consistency to this item, one cannot 
doubt the item, and inevitably concludes that relatively few mem- 
bers of the groups tested had that ability. 

Examination of current teaching practice shows that experiences 
of which this ability is a function are being provided erratically or 
not at all in elementary and secondary training. With these con- 
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ditions known, it is clear that, in so far as the sampling reported is 
adequate, the profile indicates reliability of this item as a measuring 
device, because its use results in the only type of outcome which 
possibly could be in accord with indisputable facts about function- 
ing school experiences. One can safely maintain, accordingly, that 
it is not the item which is poor; instead, the people tested were poor 
in this ability. If this item is ‘^poor” for the reasons cited, then a 
physician’s thermometer is in order and reliable only when it shows 
that a patient has normal temperature; the thermometer is out of 
order whenever the temperature it registers is not what it would 
be if conditions that function in producing temperature were 
perfect. 

Considering another aspect, assume that every fourth-grade 
child can identify a tree in a given picture. So can every fifth- 
grade child, every sixth-grade child, every seventh-grade child, and 
so on. Now according to the line of reasoning used in classifying 
the test items in question into “better” and “poorer” categories, 
this procedure is a poor device for finding out whether or not a 
child can recognize a tree in a picture, because there is no grade- 
to-grade improvement. It so happens that once one learns to 
identify in clear cut illustrations objects that are unmistakably 
trees, he has that ability to as great a degree as he could have it 
later. This ability doesn’t improve from year to year. In the case 
of such ability, uniformity of attainment is the only result that 
possibly could be in harmony with the other factor involved in the 
correlation. If ability is constant, the correlated factor — test out- 
comes — must be constant, if the test item is reliable. 

In physical tests, one does not conclude that a physician’s ther- 
mometer is a poor measuring device because all healthy children 
have the same temperature. Taking temperatures is not ruled out 
as a valuable device for testing one aspect of one’s physical condi- 
tion by reason of this uniformity. It is considered important to 
know that the child’s temperature is what it should be, if function- 
ix^ conditions are right. 
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4. Testing for map-reading ability, like testing for physical fit- 
ness, involves testing for several aspects of the matter. A child’s 
medical chart may show his vision normal, his hearing defective, 
his height and weight ^tisfactory, his temperature normal, his 
chest expansion poor, and the like. Physical condition is rated in 
terms of the number and nature of aspects in which his condition is 
satisfactory as compared with the number and nature of the aspects 
in which it is unsatisfactory. In rating map-reading ability, the 
case is similar. One must, in measuring progress in it, deal with 
comparisons of the results of the test as a whole, as well as with 
profiles of individual items. The report to the committee takes no 
account of this second means of evidencing progress. Progress most 
certainly is measured by a test as a whole, if reactions to items 1, 2, 
and 3 (known to be reliable items) are made correctly by a high 
percentage of sixth-grade children and of those of all higher grades 
and if, further, few sixth-grade children react correctly to items 
4, 5, 6, 7, and 8, while a high percentage of seventh-grade children 
and of those of all higher grades react correctly to them. This 
progress, moreover, would be evidenced conclusively even if the 
profiles of all eight items were horizontal lines. Sound conclusions 
concerning the value of a test as a device for measuring progress 
cannot be based on partial data. Progress must be viewed in terms 
of new abilities acquired at successively higher levels as well as in 
terms of an increase in the percentage of persons who have acquired 
given specific abilities. 

In view of these considerations, the following illuminating tables 
were compiled from the item profiles submitted. In considering 
them, it should be kept in mind that the sampling of several groups 
was inadequate, and that wider use of the tests might alter the 
tables. However, the high degree to which the findings accord with 
predictions based on knowledge of current learning experiences 
leads one to believe that wider sampling would not alter them 
significantly. 

In these tables is shown the performance of those tested in the 
sixth grade, the seventh grade, and the eighth grade, that of college 
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freshmen other than the Stanford and Princeton groups, and that 
of college freshmen in the latter groups. These are the only levels 
at which it is reasonable to expect progress to be evidenced gen- 
erally, and these are the only groups for which the performance of 
more than one hundred and fifty pupils was reported. In the vast 
majority of cases, geography in the high school consists of one 
semester’s work, which is elective for most students, which may 
be taken in any one of the three or four years of the high school 
course, and which deals largely in some cases with physiography, 
not geography, and in other cases chiefly with the production, 
consumption, and movement of commodities important in world 
trade — not with the geographic individualities of countries. Under 
these conditions, the only comparison with regard to Parts I and 
II which one could expect to evidence progress in high school 
would be comparison between eighth grade and freshman perform- 
ance. 

The only significant percentages in the profiles, then, are those 
for the five groups indicated. It must be remembered, too, that the 
college freshman group of the country as a whole is a more select 
group than the eighth grade group as a whole, because there is a 
tendency for the less proficient to be culled from the ranks by 
that time. Progress evidenced in terms of percentages of groups 
would tend, therefore, to be evidenced even if the freshmen had 
had no high school geography. Offsetting this, however, is the 
fact that it has been several years since most freshmen have had 
any geographic training, and that what they know evidences what 
they have retained in the interim, what they have strengthened by 
use in other fields, or what they have picked up by chance. The 
matter is by no means so simple as an item profile would make it 
seem, if one is interpreting the situation scientifically, and not on the 
ground of partial data. ^ 

Items in Part 11, Form 1, to which 75% or more of the sixth 
grade group and of groups at four higher levels reacted correctly 
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Percentages of Correct Reactions in Grades 
Item 6 7 8 13 13+ 

1 87 88 90 94 

2 90 91 (91) 95 98 

3 84 86 91 95 98 

4 85 86 93 94 98 

5 89 95 (94) 95 98 

11 85 92 92) 96 99 

13 86 93 (93) 95 97 

17 77 83 90 (90) 95 


Additional itenis in Part II to which 75% or more of the seventh 
grade group and of groups at three higher levels reacted correctly. 

Percentages of Correct Reactions in Grades 

Item 6 7 8 13 13+ 

8 T! 81 85 92 96 

14 79 88 (85) 94 

15 76 (75) (75) 88 


Additional items in Part II to which 75% or more of the eighth 
grade group and of groups at two higher levels reacted correctly. 

Percentages of Correct Reactions in Grades 

Item 6 7 8 13 13 + 

7 80 (78) 94 

10 .. 79 86 98 

25 .. 76 82 94 

26 .. 82 (82) 84 

35 .. 77 79 82 


Additional items to which 75% or more of the college freshman 
and advanced freshman groups reacted correctly. 

Percentages of Correct Reactions in Grades 

Item 6 7 8 13 13+ 

19 T T T 75 94 

21 .. .. 78 92 

30 .. .. 83 91 

48 .. 75 84 

49 .. .. 80 85 
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Additi(mal items to which 75% or more of the advanced fresh' 
man group reacted correctly. 


Percentages of Correct Reactions in Grades 


Item 6 7 8 13 13 + 


6 .. .. .. 94 

12 .. .. .. 75 

18 .. .. .. 83 

20 .. .. .. 92 

22 .. .. 88 

24 .. .. .. 80 

29 .. .. .. 81 

31 .. .. .. 79 

32 .. .. .. 84 

33 .. .. 77 

36 .. .. .. 80 

39 .. .. .. 81 

41 .. .. .. 78 

SO .. .. .. 81 


Items which failed to elicit correct responses from 75% or more 
of any group were 16, 27, 28, 34, 37, 38, 40, 43, 44, 45, 46, and 47. 
Item 23 was omitted from the eighth grade list, where it probably 
belongs, because though 77% of the eighth grade group and 94% 
of the advanced freshman group reacted correctly to it, the fresh- 
man percentage was not quite 75% — 73%. 

Tables of the foregoing types for Part I have been combined in 
the following table: 


Percentages of Correct Reactions in Grades 

Items 

6 7 8 13 13+ 


1 83 88 (87) 90 98 

S 89 90 93 (93) 97 

13 75 (75) 80 83 94 

23 75 85 (84) 90 (86) 

2S 82 86 (86) 91 95 

30 80 81 82 90 93 

34 80 (80) (75) (80) 87 
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Percentages of Correct Reactions in Grades 

Items — 

6 7 8 13 13+ 


2 80 84 95 99 

9 ' 83 (81) (78) 90 

17 76 79 88 (88) 

28 76 (75) 79 82 

38 76 79 (74) 86 

48 83 85 (80) 92 

80 78 85 (82) 89 


14 76 (73) 95 

27 .. 81 89 93 

31 .. 91 92 97 

43 .. 83 91 97 

82 .. 80 86 (78) 

92 .. 78 91 97 

97 .. 80 88 96 

98 .. 79 86 93 

100 77 82 86 


3 .. .. 85 90 

4 .. .. 85 91 

6 .. .. 75 85 

10 .. 75 86 


15 .. 81 92 

33 .. .. 81 89 

SO .. .. 77 87 

65 .. .. 81 94 

68 .. .. 92 93 

70 .. .. 76 85 

73 .. .. 75 (74) 

77 .. .. 85 95 

91 .. .. 84 95 

93 .. .. 80 96 

95 .. 92 96 


8 .. .. .. 77 

11 .. 85 

12 .. .. .. 79 

18 .. .. .. 88 

19 .. .. .. 84 

21 .. .. 81 

22 .. 80 

26 .. .. .. 78 

39 .. -• .. 87 

42 .. .. .. 90 

45 . . - - . . 76 

46 .. •• .. 84 

51 .. .. .. 82 

53 .. .. .. 87 

54 .. .. .. 80 
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Items 

Percentages of Girrect Reactions in Grades — ^Continued) 

6 7 8 13 13-1- 

56 

.... . . . . . . 86 

57 

.... . . . . . . 87 

59 

.. .. .. .. .. 81 

61 

80 

62 

.. .. .. .. .. 79 

69 

. . . . . . . . 86 

74 

. . . . . . . . 84 

78 

. . . . . . . . 83 

79 

. . . . . . . . . . 83 

81 

. . . . . . . . . . 76 

87 

. . . . . . . . . . 76 

88 

. . . . . . . . 77 

94 

. . . . . . . . 88 

96 

. . . . . . . . 86 


Items to which 75% or more of no group reacted correctly are 
7, 16, 20, 24, 32, 35 , 36 , 41 , 44, 47, 49, 52, 55, 58, 60, 63, 66, 67, 
71, 72, 75, 76, 84, 85, 86, 89, 90, and 99. Twenty-eight per cent 
of the test, then, is approximately the gap left in the best current 
achievement represented. In the case of item 37, seventy-five per 
cent of the seventh grade reacted correctly and lower percentages 
of the higher groups did so. On item 40 the eighth grade performed 
better (76%) than the two higher groups. On item 64, the eighth 
grade performed better (75%) than the freshman group, but not 
so well as the advanced freshmen. On items 29 and 83, the fresh- 
men performed better than the advanced freshmen. It is remark- 
able that such c^ses were not more numerous when one considers 
known differences in training of different groups and differences in 
the time between the date of training and that of taking the test. 
The fact that such knowledge and abilities as have been added at 
one level are retained or increased in most cases at higher levels 
certainly indicates progress. 

The percentages in parentheses indicate the only items on which 
progress is not suggested by increased percentage of right reactions 
as well as by the additions Evidenced. 

Since Part I seems reliable, it would seem from this analysis 
that children are leaving the eighth grade with relatively little 
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fundamental geographic attainment of the type for which that 
part tests. This is to be expected from analysis of current teaching 
procedures. Memorizing isolated facts does not make for ability 
to use them effectively ,^nd to think in terms of them. Whether 
the failures indicate lack of knowledge of facts or inability to use 
them in the simple new situations set up in the tests is a matter 
of little moment. What matters greatly is inability to do the latter. 
It is to be noted that; 

1. Of the nine'items concerned ,with'’|recognizing’'signs of loca- 
tion with regard to the equator (Group I items) the eighth 
grade perforihance was good with only one — the simplest 
of them all. 

2. Even the college freshmen did well with only two others 
of Group I — the next simplest two. The advanced fresh- 
men did well with three others, and three in the group 
were not handled well by any group. 

3. Of the forty-five items in Group II, the eighth grade 
handled well only one-fifth of them, identifying given con- 
ditions correctly with central Africa, the Congo, southern 
Europe, the Sahara, southern United States, eastern and 
western United States, New England, China, and Atlanta, 
Georgia. 

4. College freshmen showed little more ability of this type, 
identifying given facts, in addition, only with southernmost 
North America, northernmost South America, and Lowell, 
Massachusetts. The advanced freshman group reacted well 
on two-thirds of the items in Group II. 

5. With the other form of matching employed in items of 
Group III, the eighth grade handled well only six of the 
twenty-nine items involved — again about a fifth of the 
total. All these had to do with London or with colonizing 
countries of Europe. The college freshmen performed well 
with an additional eight, and the advanced freshmen with 
four more than the freshmen. The freshmen, then, handled 
approximately half the items of this group and the ad- 
vanced freshmen about three-fifths. 
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6. Of the sixteen items in Group IV, evidencing knowledge of 
very simple relationships, a high percentage of eighth grade 
pupils reacted correctly to seven (2, 5, 9, 23, 27, 28, and 43); 
the college freshmen to those seven and three others (6, 29, 
and 77); and the advanced freshnien also to 8, 51, and 74. 

7. The highest percentage of correct reactions to item 75, 
which calls for the mere reading of a rainfall graph, was 
69, the advanced freshman percentage. 

Summarizing, the analysis of the data furnished concerning Part 
I gives a clear, albeit distressing, picture of the paucity of geo- 
graphic outcomes which are resulting from current procedures; it 
shows which types are resulting and which are not; and it provides 
information concerning the levels at which such knowledge and 
power as do result are being acquired. 

In answering the second and third of the questions raised on 
page 261, the fourth question has been answered in essence. 
Efforts of remedial types should be directed toward filling the gaps 
in geographic achievement which the results of the tests disclose. 
Such light as the analysis sheds on the last question will be noted 
in the concluding section of the report. 

Conclusions and Recommendations 

The committee realizes fully that the tests prepared by it repre- 
sent only an initial step in the task of measuring the results of 
geographic training in elementary and secondary schools. This 
beginning, however, seems to have been effective for the purpose 
for which it was made. As to the tests themselves, four of the one 
hundred and fifty items in Form I should be revised. The difficulty 
with item 42 in Part II and the means of remedying it have been 
noted. Item 7 in Part II also should be changed because the map 
as it was printed may leave one in some doubt. Experimentation 
proved that the n^ative form of statement in items 41 and 42 of 
Part I gave some difficulty'. These are the only negative statements 
in the test and they should be made positive. It was of value to 
discover, however, that even with advanced students, negative 
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statements are confusing. Further experimentat on is needed, as 
noted earlier, with the wording of item 9 in Part II. It is possible 
that in some cases the grouping of several items under one state^ 
ment in Part I led those tested to resort to a multiple choice form 
of selection instead of reacting to each numbered item individually, 
as they were told to do in the initial directions. If this proves, 
upon further experimentation, to constitute a difficulty, it would 
be advisable either to make a separate statement for each, or to 
stress even more in the directions the fact that all the items 
under a given statement may be true, that all may be false, or 
that some may be true and others false. 

After such revisions as seem appropriate in the light of the fore- 
going comments are made, further data should be collected from 
the use of the tests. The use should be such that adequate data 
for evaluating them as to reliability will result therefrom. While 
the data furnished to the committee do not show unreliability, 
they are not sufficient to do more than indicate considerable 
reliability. In order to handle statistically and scientifically both 
types of factors involved in establishing reliability, scores must 
be compared with intelligence test scores, reading scores, and the 
like, and with data concerning the specific amount and nature of 
the geographic training of those tested. In cases where there are 
not already strictly comparable items in two or more forms, such 
items should be supplied. It would be highly desirable to construct 
a battery of tests, one dealing with each phase of ability. A test 
of twenty items all dealing with the ability to read latitude from 
various types of projections, for example, could be so constructed 
and used as to establish beyond any doubt the value of items used 
for this purpose. 

Judging from the results secured from the use of Parts I and II, 
it seems just as well that the expense of administering similar tests 
for Parts III and IV was not borne at this juncture. Since there is 
fully as much emphasis on mere fact memorization in courses 
currently administered in "Economic Geography" and “Principles 
of Geography” as there is in those in elementary geography, the 
findings undoubtedly would have been of the same general type. 
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Unsatisfactory achievement would have been revealed and it 
probably would have been practicable, as in the case of Parts I 
and II, only to have collected a small part of the data needed for 
their thoroughgoing evaluation. However, as soon as it is practic- 
able to do with Parts III, IV and V what has been suggested with 
regard to the use of Parts I and II, they should be constructed and 
so used. 

In constructing Parts III and IV, the committee planned to 
proceed much as with Part I. In Part III, instead of matching 
geographic characteristics with given regions, facts about the 
nature, conduct, and pattern of a given activity were to be identi- 
fied with the activity by the use of matching devices similar to 
those used in Groups II and III, Part I. Items in Group IV, Part I, 
were to have had their counterpart in items which stated given 
aspects of an activity and called for the selection of human and 
natural conditions with which they are associated. The geographic 
characteristics of trade regions, ports, and trade routes, regions of 
production, and regions of consumption were to have been identi- 
fied with specific trade, production, and consumption regions 
instead of with countries. In Part IV, corresponding groups of 
exercises were to have been provided for identifying various human 
and natural facts with given types of topography and topographic 
regions, different types of climate and climatic regions, and the like. 
No test gives a comprehensive view of fundamental geographic 
achievements if it does not test for all the major phases of such 
achievements. 

It seems that the portions of the tests completed scarcely could 
fail to call additional attention (1) to types of outcomes which are 
of basic importance in geography, and (2) to the fact that many 
of these outcomes are not resulting from procedures now in common 
use. If this is the case, two of the chief purposes of the test will 
have been served. 

It also seems clear that the testing devices used have much 
reliability and with further work along the lines indicated will 
constitute even more effective tools of measurement. To those 
who derive ccHnfort from comparing accomplishments, not with 
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what it is practicable and valuable to accomplish, but instead with 
what other folk have accomplished, the analysis of results affords 
a basis for the purpose. A given sixth grade performance can be 
compared with that of the sixth grade group tested, a given seventh 
grade performance with that of the seventh grade group tested, 
and so on. 

So long, however, as one bases his procedure solely on what has 
been accomplished generally in the past his efforts toward improve- 
ment resemble trying to elevate one’s self by tugging at one’s 
bootstraps. If, instead, one sets achievements which are the best 
instead of the most general as his goal, his efforts resemble the use 
of a ladder in climbing. Though tests based on fundamental, prac- 
ticable achievements show gaps in the outcomes of past training 
in geography which are not pleasant to contemplate, the results 
of their use should be a challenge, to all concerned, to direct efforts 
toward filling them. Only through effort directed toward achieving 
the best can many of the potential contributions of geography, 
the effective pursuit of social studies become actual contribution#. 
It is to the interest of those concerned primarily with the social 
studies, then, as well as of those concerned primarily with geog- 
raphy, to encourage and promote efforts directed to the improve- 
ment of geographic training. 
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SUMMARY OF REPORT UPON THE ITEM STUDY OF THE CALKINS- 
PARKER GEOGRAPHY TESTS 

Truman L. Kelley 

The full report by the writer upon the study of the items com- 
prising the geography test, which is referred to by Miss Parker, 
contains item profiles for 300 geography test exercises. These call 
for far too much space to permit of their reproduction here, 
so that the essential portions of the report and a sample only of 
these item profiles are here given, following the next paragraph. 

After a careful reading of the report by Miss Parker, the writer 
notes with regret that there is evidence of unfortunate misunder- 
standing as to the nature of evidence upon reliability and validity. 
Nevertheless it is not such as would lead him to modify his ap- 
praisals as to reliability and validity based upon the item study. 

The basic data from which this report is made are the item pro- 
files and table herewith of reliability coefficients. The reliabilities 
reported are those pertaining to single items of the test. They 
were obtained by splitting the items of a given class into halves, 
of, say, m items each, and correlating the halves for the groups as 
described. The result is a reliability coefficient for a test of m 
items. Then by means of the Spearman-Brown formula the re- 
liability coefficient for one item has been estimated and is the 
figure given in the table upon the following page. 

The item profiles show much absence of improvement from 
grade to grade in the abilities measured by a majority of the items, 
and they also show a large number of mean scores of items of the 
same order of magnitude as chance scores. Three sample profiles 
are here shown, (1) the “best” item, (2) an item of median excel- 
lence, and (3) the “poorest” item, of Form I, Part I. 

Where either of these two facts is established for an item, 
it indicates that the item will not function as a measuring device 
for the grades in question. The items which fail to show appre- 
ciable grade-to-grade impitovement are those listed as “poorer” 
items. Reference to the table shows that, with the exception of 
Form I, Part II, Stanford and Princeton Freshmen (where the 
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TESTS AND MEASUREMENTS 


ITEM PROFILE 

CALKINS-PARKER GEOGRAPHY TEST 
Form I, Part I 



99 

96 

93 

90 

87 

84 

81 

«• 78 
U 75 
i 72 

I 

£ 66 
tS 63 
£ 60 
57 


56 58 


c£ ^7 

Ar\ 


18 Characterized by T. L. Kelley as an excellent item. 


*Stanford snd PriAceton freshmen. 


item reliability is .052); Form II, Part II, True-False, Grade 9 
(where the item reliability is .070); Form I, Part II, Multiple 
Choice, Graduate Students (where the item reliability is .120); 
and Form II, Part II, Multiple Choice, Grade 13 (where the item 
reliability is .087), and Graduate Students (where the item relia- 
bility is .131), these items fail to measure anything consistently 
for the grade in question^ It is necessary to conclude that these 
items are not functioning as measuring devices for the elementary 
school, for the high school, and for the university, except in the 
exception^ cases noted. 
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ITEM PROFILE — Continued 


c 

o 

a 


c 

QJ 


39 

36 

33 

30 

27 

24 

21 

18 


89 


84 


75 


77 


Grade 6 7 8 9 10 11 12 13 14 IS 16 • 

No. of Cases 225 301 193 131 74 92 26 968 13 3 74 465 

_ 

96 
93 
90 
87 
84 
81 
. 78 

£ 75 

72 
69 
66 
63 
60 
57 
54 
51 
48 
45 


76 


67 


66 


68 


69 


58 


44 


Characterized by T. L. Kelley as a mediocre item. 


*Stanfo[(l and Piinceton freshmen. 


Though the “better” items show grade-to-grade improvement, 
this is not marked, and the item reliabilities remain so low in the 
7th grade and in the 9th grade (except for Part II, Multiple 
Choice) that even these items can hardly be said to be serviceable 
in measuring individual or class differences at the elementary and 
junior high school levels. The “better” items, and as noted some 
of those listed as “poorer,” do function to establish group and 
even individual differences at the university level, though Part I 
items are not functioning very efficiently. 

It may be concluded that the test, when the better items only 
are retained, could be used to establish the inability of pupils as 
now found to cope with such problems, but it could not be used 
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ITEM PROFILE — Continued 


Grade 6 7 8 9 10 11 12 13 14 IS 16 • 

No. ofCa»M 225 301 193 131 74 92 26 968 13 3 74 465 


100 


83 


73 


75 


78 


74 


73 


73 


69 


80 


67 


59 


33 Characterized by T. L. Kelley as an item having no diagnostic power. 

*Stanfor<] and Princeton frtahmtn. 


to establish individual differences as between pupil and pupil, nor 
even in any very serviceable way as between class and class at 
the junior high school level. It may well be that the items show- 
ing the shortcoming mentioned do measure a function which it is 
important to teach, and which, having been taught, could then be 
measured by means of these items. No evidence at hand confirms 
or refutes this hypothesis. The evidence indicates that as geog- 
raphy instruction now stands and as sampled by our testing pro- 
gram, the items do not reveal differences in ability. 

Part //, Multiple Choice: The “better” items of the multiple 
choke test yield low but serviceable reliabilities for purposes of 
group measurement. In the 9th grade the item reliabilities are .052 
(the average of .057 and .047). To yield a serviceable group test, 
the retiability should be .40 or better. Fifteen of the “better” 
Aiultirde dioke items will give a test of reliability .45, and as S3 
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such are available, it is possible to build up three forms of 15, 16, 
17, or 18 items each, which would be serviceable for group measure- 
ment at this grade level. Probably the 18 item test would also be 
serviceable at the 7th grade level, as, judging by Part I, the re- 
liability at this level will probably not be greatly different from 
that found at the 9th grade level. 

Part II, TTiie~False: The items of this part show no promise of 
yielding a group test for grades 7 and 9 under present conditions 
of geography instruction. It is probable that a serviceable group 
measure at the university level exists in these items, but as Form 
II, Part II, T. F., was not given at the university level no differen- 
tiation of the better and poorer items for the true-false part of 
this form is possible. 

Form I, Part I, True-False: The average item reliability of the 
“better” items is .034 (average of .0141, .025, and .039). We 
therefore require a test of 23 items to yield a reliability of .45, As 
there are some 83 of these “better” items in Form I (Form II, 
Part I has not been tried out), it is entirely feasible to build up 
three forms of 25 items each which would be serviceable for group 
measurement. These forms would be serviceable for such meas- 
urement in the junior high school, the senior high school, and at 
the university level. The three together would yield a reliability 
of .71 which is scarcely reliable enough for purposes of individual 
diagnosis. 

This report is complete for the data at hand, but due to certain 
gaps in the data, particularly as it concerns parts not tried out, 
and in view of the fundamental issue as to the value and place of 
a test which may measure important functions not now taught, 
but which does not well measure present attainment of pupils, no 
recommendation is made as to the next steps to be followed. 
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THE CONSTRUCTION OF EXERCISES IN THE 
USE OF HISTORICAL EVIDENCE 

By Marion Clark 

Director of Upper Elementary and Junior High Schools, 
Montclair, N. J. 

This statement will cover six phases of the work done in the 
construction of two groups of exercises in the use of historical 
evidence, an elementary group and an advanced group: (1) the 
definition of the purpose and range of the tests, (2) the selection 
of materials suitable for these exercises, (3) construction and forms 
of the exercises, (4) developing directions for testing, (5) the test- 
ing of 2000 pupils and students, (6) statistical results. 

The Exercises in the Use of Historical Evidence were made under 
the direction of Professors Truman Kelley, Henry Johnson, and 
A. C. Krey, during the years 1928-1930. 

1. The specific purpose of the exercises is to test the ability to 
make legitimate use of a given piece of historical evidence, and of 
that only. The scope is very wide, since it includes historical 
material from the beginning of time to the present. The range is 
also wide, since the exercises are designed to test the ability to use 
evidence, from the third grade level to the graduate college level. 

2. The first six months were spent in the collection of material 
selected from historical sources, from the works of historians, from 
maps and pictures. Certain difficulties and limitations were met. 

First, all questions must necessarily lend themselves to a system 
of checks as answers, in order to make possible the scoring of 
the papers on an objective basis by clerical assistants. 

Secondly, as far as possible, varying degrees of acceptability in 
answers must be avoided. This called for an absolutism of response, 
which is difficult to arrive at in the field of the social studies. 

Thirdly, the vocabulary must be suited to successive grade 
levels. 

The sidection of material for the third grade, the lowest level 
to h4,tested, offered exceptional difficulty on account of the limited 
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reading ability of the children. Even the Thorndike Word List 
was inadequate as a guide, for the reasons that it shows the fre- 
quency of words rather than their ease of reading and that it does 
not include on a third grade level many words such as Eskimo and 
colonial^ which become familiar to children through social studies 
activities, while airplanes^ of course, does not appear in it at all. 
Also, it is difficult to make exercises on this level which call for 
some simple type of historical thinking beyond the mere power 
to read and interpret. 

The selection for the highest levels offered the problem of arrang- 
ing adult materials on levels of increasing difficulty. 

By October of 1929 a sufficient number of exercises had been 
worked out to call for approximately 2,000 responses. In the first 
form all of the exercises called for conclusions to be drawn from 
evidence. Critical analysis and suggestions for reorganization of 
the material were then undertaken. Professor Henry Johnson, and 
others in the field of history, suggested the rejection or revision of 
some material improperly used as evidence. Dr. Truman Kelley 
criticized the exercises from the standpoint of their suitability as 
testing material; and a number of laymen who read the exercises 
critically offered helpful suggestions. 

3. As a result of conferences with Professors T. L. Kelley, 
Henry Johnson, and A. C. Krey, the exercises were recast into 
two distinct tests; the first for children on an elementary school 
level from the third through the sixth grade, the second for stu- 
dents in the junior high school, high school, and college. 

The advanced test was divided into four groups: Group I was 
designed to test the ability to draw conclusions from evidence; 
Group II was designed to test the ability to discover one fact 
from another known fact; Group III was designed to test the 
ability to group evidence that is pertinent to a problem; Group IV 
was designed to test the ability to discriminate among types of 
evidence. 

An attempt was made to organize the elementary test into the 
same four groupings. However, no terms equivalent in meaning to 
“general fact,” “specific fact,” and other types of evidence^ were 
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found within the vocabulary possibilities of third and fourth grade 
children. Therefore, no Group IV was developed in the elementary 
exercises. 

To provide for historical thinking for each response, it was desir- 
able to lessen the possibilities of guessing. This was secured by 
two means: (a) the use of three, instead of two, possible choices 
for each response; (b) the use of an unequal number of statements 
wherever two lists were to be matched. 

The tests in this form were criticized again by Professor Henry 
Johnson and reorganized. 

Before the exercises were published it was necessary to develop 
directions for pupils, to be printed at the beginning of each exercise. 
Vocabulary difficulties were numerous. Several months were spent 
in trying out various wordings. The elementary tests were tried 
by twenty individual children who commented upon their under- 
standing of the directions. The advanced test was given to several 
adults of outstanding critical keenness, in order that inconsistencies 
and irrelevancies might be discovered. Among these adults were 
Miss Dorothea Marston, Director of the Child Guidance Depart- 
ment of Montclair, New Jersey; Miss Margaret Mathias, author 
of Beginnings of Art in the Public Schools, Art in the Elementary 
Schools and The Teaching of Art; Miss Pearl Monks, principal of 
Wade Park School, Cleveland, Ohio; and Dr. Mary Reed of 
Teachers College. 

A final dbnference with Dr. A. C. Krey and Dr. Kelley on 
October 4, 1930, resulted in further revisions, and the decision 
that the exercises should be printed for experimentation. The tests’ 
at that time consisted of: 


ELEMENTARY EXERCISES 



Group 

Number of 
Exercises 

Number of 
Responses 
Calkd for 

I. 

l^rawing Conclusions from Evidence. . 

8 

37 

11 . 

Maiung I'roper Implications from 
Facts 

5 

18 

III. 

Grouping under General Heads 

S 

34 


*Tlw tests from which the tables accompanying this paper have been drawn 
are published by Chariea Scribnei^s Sons. 
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ADVANCED EXERCISES 



Group 

Number of 
Exercises 

Number of 
Responses 
Called for 

I. 

Drawing Conclusions from Evidence. . 

9 

SS 

II. 

Making Proper Implications from 
Facts 

8 

22 

III. 

Grouping under General Heads 

7 

47 

IV. 

Classifying Types of Evidence 

6 

30 


Examples of these exercises with a study of statistical results are 
included at a later point in this article. 

4. The next step in the procedure was to develop directions for 
giving the tests. Two thousand booklets of each group of exercises 
had been printed. The exercises were given to four high school 
classes, four junior high school classes, and six elementary classes 
by testers who adapted their explanations to the needs of each 
class as the tests were given. A stenographer took careful notes of 
all that was said by the tester, questions that were asked by the 
children, and observations in regard to various types of difficulties 
that the children had with the questions, exercises that the children 
especially enjoyed and any other observations that might be of 
help in organizing the final directions for giving the test. From 
these notes, directions for giving the tests were drawn up, sub- 
mitted to Dr. Kelley, and revised upon the basis of his criticisms. 

5. The remaining tests (approximately 1,500 of each exercise) 
were given with the assistance of the following groups: Professor 
Henry Johnson of Teachers College, Columbia University; Mr. 
Frank G. Pickell and the principals and teachers of the Montclair 
Public Schools; Professor Roy Hatch and Miss Florence Stryker 
of the State Teachers College at Montclair; Miss Anna Gannett, 
principal, and the teachers of Brookside School in Montclair; Miss 
Clara Levy, Miss Eda Willard and Miss Rachel Jarrold, heads of 
the departments of history of the state normal schools of New 
Jersey at Newark, Glassboro, and Trenton, respectively; Mr. Har- 
old Ferguson, principal of the high school in Montclair and Miss 
North, the head of the department in history in that high school; 
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Mr. William Davis, supervising principal of Haddon Heights, New 
Jersey; Mr. Charles Beek, principal of the junior high school in 
Summit, New Jersey; Miss Jane Gilbert of the Buckley Schools in 
New York; Sister Mary Boniface, School of the Holy Child, Sum- 
mit, New Jersey; Mr. Guy Baskerville, University School, Pitts- 
burgh, Pennsylvania. 

6. The papers were sent to Dr. Truman Kelley for examination 
and statistical analysis. 

The number of pupils used in the various grade levels are; 


Grade 3 — 176 pupils 
Grade 4 — 261 pupils 
Grade 5 — 266 pupils 
Grade 6 — 297 pupils 
Grade 7 — 108 pupils 
Grade 8 — 107 pupils 
Grade 9 — 110 pupils 
Grade 10 — 23 pupils 


Grade 11 — 14 pupils 
Grade 12 — 22 pupils 
Grade 13 — 482 pupils 
Grade 14 — 57 pupils 
Grade 15 — 35 pupils 
Grade 16 — 21 pupils 
Grade 17 — 21 pupils 


’ For convenience, the detailed discussion of the exercises and sta- 
tistical analysis is divided into two parts: 1. The discussion of the 
elementary exercises. 2. The discussion of the advanced exercises. 


Elementary Exercises 

For purposes of illustration, exercises have been selected con- 
taining items that showed promising results, and items so lacking 
in promise that they were discarded. 


Group I 
Exercise IV 

This is the only book the little children in America had in early 
colonial schools. It was called a horn book. On a thin piece of 
wood) usually about fo^r or five inches long and two inches wide, 
was placed a sheet of paper a trifle smaller than the wood. At the 
top the alphabet was printed in large and small letters. Below* were 
sim^ syllables such as ab, eb, ib, etc. Then came the Lord’s 
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Prayer. This printed page was covered with a thin piece of yellow- 
ish horn which showed the letters through so they could be read. 

After you have looked at the picture and read the story about it 
carefully, place a check in the proper square after each statement. 





True 

Does not 
Tell 

False 

l. 

The children in colonial schools did 
not learn to read 

n 

□ 

□ 

2. 

The people in colonial days thought 
it was important for children to 
learn their letters. 

□ 

□ 

□ 

3. 

The paper used in the horn books was 
made in America 

□ 

□ 

□ 

4. 

Many interesting stories were written 
for the boys and girls in the Ameri- 
can colonies to read in school 

□ 

□ 

Q 

5. 

There were printing presses in those 
days 

□ 

□ 

□ 
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The profiles of the items in this exercise follow: 

ELEMENTARY— GROUP I 
EXERCISE IV 


Item 1 Item 2 Item 3 Item 4 Item 5 

Grades Grades Grades Grades Grades 


3456 34563456 3456 34S6 

99 

96 

93 

90 .. 

87 

84 

81 

78 79 79 

75 .... 74 

72 

69 68 

66 

63 

60 ..61 

57 56 55 

54 

51 52 

48 48 

45 43 43 

42 41 

39 40 

36 
33 
30 
27 
24 


21 19 .. .. 

18 ' 17 

IS 

12 11 .. 

9 8 


Mean per 

cent correct 255/4 223/4 195/4 145/4 65/4 


Sum of mean per cents coiTect=883/20«»44.15 

In the case of items 1, 2, 3 and 4, the improvement from grade 
to grade is evident and regular. Item 5, however, shows a lower 
number of correct responses, on each successively higher grade 
level. This is somewhat puzzling, since one would expect children 
to assume that printed pages used in school, proved the presence 
of printiiig presses of some sort. The suggestion is that thoughtful 
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30 


27 
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children would decide that there was nothing to correspond with 
our present printing press. It is, perhaps, even more likely that 
the children looked for a definite statement that there was such 
a press, and, not finding it, marked the answer, *‘Doe$ not tell.*’ 
The item could not be included in the revised form of the test. 

Group II 

The behavior of the items in all the exercises in this group was 
so good, that I have chosen to present here an overlapping exercise, 
that is, one which was used in both the elementary and the 
advanced tests. 


EXERCISE IV 

These facts all concern the makina of the first railroad across the United 
States. It was built by two companies, the Union Pacific and the Central Pacific. 
Fact I. Congress gave about 232,000 and a large piece of land for each mile 
of railroaa which either company built. 

Fact II. One day in 1869, 4,000 laborers, mostly Chinamen, laid more than 
ten miles of track. 

Fact III. Sometimes Indians surprised the builders and drove them away. 

Fact IV. Not long after the railroad was finished herds of buffalo were no 
longer seen crossing the tracks, but great herds of cattle grazed on the 
plains instead. 

Put a check in the proper square under the number of each given fact which 
helps you to know the first statement. Do the same with each of the other state* 
ments. 




Fact I 

Fact II 

Fact III 

Fact IV 

1. 

Congress was not trying to help 
one company to beat the other. . 

□ 

□ 

□ 

□ 

2. 

In 1869 Chinese laborers were not 
forbidden to come into the United 
States as they are now 

□ 

□ 

□ 

□ 

3. 

The United States government 
thought it was very important 
for the east and the west to be 
joined by a railroad 

□ 

□ 

□ 

□ 

4. 

The railroads were built under 
great difiSculdes 

□ 

□ 

□ 

□ 

S. 

Some of the people who moved 
west undertook cattle raising... 

□ 

□ 

□ 

□ 


The behavior of Item 3 is given here in detail to indicate the 
general type of result .obtained in this exercise; namely, an excellent 
profile through the ninth grade, but no discriminating value on the 
higher levels. 
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ELEMENTARY— GROUP 11, EXERCISE IV, AND 
ADVANCEI>-GROUP II, EXERCISE I 


99. 

96. 

93. 

90. 

87. 

84. 

81. 

78. 

75. 

72. 

69. 

66 . 

63. 

60. 


iTIM 3 


Grade 


3 4 5 6 7 8 9 10 II 12 13 14 15 16 17 


94 .. .. 


84 


89 

.. .. 87 — .. .. 

86 86 = 86 


78 


76 


71 


57. 

54. 

51. 

48. 


58 


45. 

42. 

39. 

36. 

33. 

30. 

27. 

24. 


47 


30 


21 . 

18.. 

15., 

12 .. 


23 


11 


Mean per cent correct *.986/ 15 


Some of the items in this exercise showed a good profile only 
through the sixth grade. No items had differentiating value for 
high school or college students. 

f 310 ] 




ELEMENTARY BXERC1BE8 


Dr. Kelley’s reports on Group II of the Elementary Exercises 
vras as follows: *'As all of the exercises look promising, the relia- 
bility of the entire test for 100 fourth grade pupils was calculated. 
The test was split in two parts, Exercise II and IV being one part, 
and I, III, and V the other. The correlation between these parts 
is .527 so that the reliability of the entire test is .69. To secure 
a group test of reliability .40 or better, two exercises having a total 
of 6 items or more are required. We therefore need but 4 of the 5 
exercises in order to secure two forms of the desired reliability. 

I suggest that Form I be made up of Exercises II and V, and Form 

II of Exercises III and IV. The reliability of either of these in the 
fourth grade is approximately .48, which is very satisfactory for 
a group test.” 

Group III 

All but one of the 34 items in the five exercises in this group 
showed good profiles, although Exercises I and 11 were too easy, 
as shown by a profile range approximately from 72 to 97 per cent 
correct. 

Exercises IV and V were used as overlapping tests, being the 
last two of the elementary exercises and the first two of the 
advanced exercises. 

Exercise IV is given here: 

If the Spanish king had tried to prove in 1620 that he owned all of North 
America, which of these facts could he have used? Place a check in the proper 
square at the right of each statement to show whether he could or could not. 


He could He could not 
have used have used 

these facts these facts 


1. In 1493 the Pope declared that all the land west of 

a cenain line belonged to Spain. This gift in- 
cluded all North America 

2. Columbus touched the mainland of America in 

1598 

3. The English king in 1606 gave the land along the 
eastern coast of North America to two companies, 
the London Company and the Plymouth Company. 


□ 

□ 

□ 


□ 

□ 

□ 


{Continued on following page) 
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{Coniinu/d from prevtous page) 


He could 
have used 
these facts 


4. Columbus discovered America and claimed it all 

for Spain in 1492 

5. In 1609 Henry Hudson discovered the Hudson 

River and claimed it for Holland 

6. The English king said that England owned the land 

because Tohn Cabot had discovered it about 100 
years belore 

7. In 1524 Verrazano sailed along the coast of North 

Americai entered New York harbor, and claimed it 
for France 


□ 

□ 

□ 

□ 


Item I 


He could not 
have used 
these facts 


□ 

□ 

□ 

□ 


Grades 

3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 


99. 

96. 

93, 

90, 

87. 

84. 

81. 

78. 

75. 

72. 

69. 

66 . 

63. 

60. 

57. 

54. 

51. 

48. 

45. 

42. 

39. 

36. 


95 95 95 95 

93 

92 

.. .. m 89 .. .. 


77 


78 


70 70 

= 69 — 


.. 54 .. . 

(chance score) 


34 


33 


Mean per cent correct =1170/15 
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ELEMENTARY EXERCISES 


The profile of Item I is an illustration of an item that showed 
differentiating power from the third through the twelfth grade. 

A summary of the findings on the elementary tests, and the 
recommendations for the^making of the revised tests into Form 
A and Form B, are also given. 

ELEMENTARY EXERCISES 
(1,000 pupils — Grades 3 to 6 inc.) 


Group 

and 

Exercise 

No. of Items in each 
Exercise 

Mean per cents 
correct 

No. of Items showing 
fair or good profiles 

Items recommended 
for 

Revised Form A 

Items recommended 
for 

Revised Form B 

Reliability of two 
forms found by 
correlating scores 
from Too 

4th grade pupils 

Group I 







Ex. I 

S 

60.35 

5 

All 



Ex. 11.... 

s 

65.05 

5 


All items 


Ex. III... 

3 

48.00 

2 



.500 

Ex. IV... 

5 

44.15 

4 

l, 2, 3, 4 


(“A -3.34; “B-3.95) 

Ex. V 

S 

55.05 

2 




Ex. VI... 

S 

49.75 

3 




Ex. VII., . 

S 

62.63 

5 


All items 


Ex. VIII.. 

4 

48.73 

1 




Group II 







Ex. I 

3 

79.17 

3 




Ex. II.... 

3 

66.25 

3 

All items 



Ex. III... 

3 

73.42 

3 


All items 

.48 

Ex. IV. . . 

5 

81.09 

5 


All items 


Ex. V.... 

4 

78.13 

4 

All items 



Group III 







Ex. I 

8 

86.96 

8 




Ex. II.... 

7 

87.78 

7 

All items 


.431 

Ex. III... 

7 

77.14 

7 


All items 

{“A-3.2S; “B-3.0S) 

Ex. IV. . . 

7 

76.34 

6 

All items 



Ex. V.... 

5 

73.60 

5 


All items 








The exercises, reset into Forms A and B according to these 
recommendations, have been published by Charles Scribner* s Sons 
and are available through the publishers or through the American 
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Historical Association Investigation of the Soda! Studies in the 
Schools. 

Advanced Test 

Group I {Drawing Conclusions from Evidence) 

Two complete exercises are given here; Exercise IV because its 
results illustrate various unpromising situations; and Exercise 
Vlll because it illustrates hopeful possibilities. 

EXERCISE IV 

Durin{[ the Revolutionary War which wai fought to defend the independence 
of Amenca, the American army wa« often without proper auppliei. In 1777, on 
the 22d of December, George Washington, the comtnanner of the American army, 
was prevented from attacking Howe because two brigades of soldiers had refused 
to fight until they had food. They had been without bread and meat for two or 
three days. 

Neither did the soldiers have proper clothing. When they marched into winter 
quarters in 1777, their feet were bloody. 

During the same year the farmers in Pennsylvania had plenty of grain and 
vegetables to sell. At the very time when the American soldiers were suffering 
from winter cold in Valley Forge, there were hundreds of pairs of shoes, stockings 
and clothing lying in the woods for want of teams to draw them to the army and 
money to pay teamsters. 

Congress, a body of men chosen by the different states, had the following plan 
for providing for me army. They appointed two commissary*^enera]s, one to uke 
charge of purchasing supplies and one to take charge of distnbuting them. How- 
ever, the men who did the buying and transporting were responsible, not to the 
commissary-generals, but directly to committees in Congress. These committees 
discussed the matter and decided what money should be allowed and gave orders 
for carrying on of the work. 

After you have read this paragraph carefully, place a check in the appropriate 
square after each statement. 



True 

Not proved 
by evidence 

False 

1. That Congress made no effort to supply 

□ 

□ 

□ 

army with food and clothes. 

2. That the ornnization of Conjp?e$8 into 
committees did not promote emdency in 
the army. 

□ 

□ 

□ 

3. That there were no supplies to be ob- 

□ 

□ 

□ 

tained in the colonies. 

4. That Congress was entirely responsible 




for theditoouraging conditions in Wash* 
ingtea'i atiay. 

□ 

□ 

□ 


'Hie profiles of Items 1 and 2 follow: 
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ADVANCED EXERCISES 


ADVANCED— GROUP I 
EXERCISE IV 


Item 1 » Item 2 

Grades 

7 8 9 10 11 12 13 14 IS 16 17 789 10 11 12 13 14 IS 16 17 


100 100 

99 


97 97 

96 ..96 % 


— 9S 9S — 9S 

93 94 — 93 


90 TT 


87 




51 


48 

48 .. 

4S 


42 

43 

39 


36 

36 36 



30 


27 


24 


21 . 

n 

i 

18 1 


IS 

IS .. 15 

12 


9 .. 

n 

6 


3 

4 

Mean per cent correct 

1057/11 

281/11 
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The profile of Item 1 shows clearly that the question is too easy 
for these levels. In a new try-out of these questions Item 1 might 
be stated: That Congress was more interested in politics than in 
supplying the army with food and clothes. 

In this case the correct response would be, “Does not tell.” It 
seems probable that this might not be so easy. 

The profile of Item 2 shows sufficient difficulty and some dis- 
crimination of grade, but the results are so erratic that they 
suggest a weakness in the item. It would seem wise before trying 
the exercise again, to submit this item to the criticism of several 
history experts and to reword the item on the basis of their judg- 
ments. 

The profiles of Item 3 and Item 4 are somewhat different from 
those of 1 and 2. 

Item 3 has differentiating value on all three levels, junior high, 
high, and college, and could be retained in resetting the test. The 
peculiar behavior of the item on the eleventh and seventeenth 
grade levels requires a detailed re-examination of the papers and a 
study of causes. 

Item 4 shows fair differentiating power up to the college level. 

It is evident that Exercise IV should be reshaped and that fur- 
ther experimenting should take place. The nature of the changes 
which seem to be called for are discussed in a later paragraph. 

EXERCISE VIII 

In the year 1800 Spain ceded to France, New Orleans and the mouth of the 
Mississippi. Thomas Jefferson became president the following spring. He believed 
that this would interfere with the trade of the United States through the Mississippi, 
River. He appointed Mr. Livingston and Mr. Monroe to try to buy New Orleans from 
France. Napoleon, then dictator of France, offered to sell United States the land 
which Jefferson wanted and also the great tract of land called Louisiana. The 
powers of the government are defined by the constitution. 

Three clauses in the constitution were interpreted by some people as covering 
the p^er of die United States to buy Louisiana. 

Article 1, Section 8. — "The Congress shall have power to lay and collect Taxes, 
Dudes, Imposts, and Excises, to pay the Debts and provide for the Common De- 
fense and generd Welfare of the United States." 

• « * 

*To make aO Laws which shall be necessary and proper for carrying into Execu- 
don the foregouw Powers, and all other Powers vested by this consdturion in the 
^vemment of the United States, or in any Department or Offices Thereof." 
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ADVANCEEI-GROUP I 
EXERCISE IV 


Item 3 Item 4 

1 Grades 

7 8 9 10 11 12 13 14 IS 16 17 789 10 It 12 13 14 IS 16 17 


100 

99 


98 .. 

96 “ 97 . . 


93 


92 

90 = 


87 87 


86 

84 


83 

81 .... — 81 


79 

78 — 78 

79 

7S 


72 


71 

69 


66 


63 


60 


57 

S8 

= .. .. 57 .. 

S4 ... 


51 

52 

48 

49 

4S 


42 


39 

40 

. . 39 “ 

36 

36 

33 

35 

30 

30 

Mean per cent correct *» 

9S2/tl 

-531/11 
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Article IV, Section 3. — "The congreet shall have Power to dispose of and make 
all needful Rules and Regulations respectiDg the Territory or other Property be- 
longing to the United States.” 

President JeflTerson did not believe that these clauses gave the government the 
right to add new land. He said he thought that it was "Safer not to permit the 
enlargement of the Union but by amendment to the Constitution.” 

When the treaty by which all of the land mentioned above came to Congress to 
be ratified, Jefferson wrote, “I suppose they (Congress) must then appeal to the 
nation for an additional article to the Constitution approving and confirming an 
act which the nation had not previoudy authorized.” 

Jefferson prepared an amendment to the Constitution permitting the purchase 
of land. He thought this should be adopted. Some of the Senators did not think 
that was necessary but Jefferson said, ‘^ur peculiar security is in the possession 
of a written Constitution. Let use not make it a blank paper by construction.” 

The day after he wrote the above quotation, Jefferson had a letter from Mr. 
Livingston in Paris urging him to hurry die ratification of the treaty. Then Jef- 
ferson wrote another letter in which be said, “The less we say about constitutional 
difficulties respecting Louisiana the better.” 

Within four days the treaty was ratified and the land was purchased, though no 
change had been made in the Constitution. When the president presented the 
treaty to Congress to be voted upon, he did not mention the constitutional diffi- 
culty. Later on, Senator J. Q. Adams proposed an amendment to the Constitution 
to authorize the addition ofland. It was not seconded and has not been heard of 
since. 

After you have read these paragraphs carefully, place a check in the appropriate 
square after each statement. 

All the items, except Item 4, show power and might be retained. 
However, this exercise, in common with most of the exercises in 
Group I, show clearly that three problems must be taken into 
consideration before resetting the exercises. The first problem is 
that of setting up a series of items based upon given evidence, 
each item of which will show improvement in response from grade 
to grade, over a range of ten grades; that is, from grade VII to 
grade XVII. Certain probabilities appear: that the evidence, and 
the given items, difficult enough to challenge college students, 
would be so far beyond the reading comprehension of seventh and 
eighth grade pupils that the element of guessing would enter. The 
reverse is also true; that material easy enough for junior high 
school pupils to comprehend and yet difficult enough to test the 
power of college students to draw conclusions, is hard to find. 
The attempt to formulate the more difficult items for these exer- 
cises has resulted in some that are obtuse rather than clear and 
difficult. Thus the complication of various interpretations by ad- 
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advanced exercises 


vanced students is added to the possibility of guessing by the 
junior students. 

In remaking the test, there are two possible courses to follow. 
One possibility would be to divide the exercises into two parts, 
limiting the scope of each, one for grades seven to eleven, inclusive, 
retaining much of the present material, and one for grades twelve 
to seventeen, inclusive. The other possibility would be to expect 
each exercise to show differentiating power from item to item, 
rather than within each item. 

The second problem is that the difficulty of shutting out all 
evidence except that bit which is presented, apparently increases 
with increased education. Some of the responses to items indicate 
that such is the case, and discussions with high school and college 


' ■ " wy — ■ 

True 

Not proved 
by evidence 

False 

1. The framers of the Constitution thought 




that United States in 1783 was as Urge 
as its successful development would ever 
permit. 

□ 

□ 

□ 

2. They did not think about the possibilities 




suggested by large pieces of land belong- 
ing to foreign countries lying next to 

□ 

□ 

n 

United States territory. 

3. In case territory was to be added to the 




United States an interpretation of the 
Constitution or an amendment to it 

□ 

□ 

□ 

wou Id have to be made. 

4. JefFerson was insincere either in his be- 




lief that the purchase was unconstitu- 
tional, or in his oath of office to support 
the Constitution. 

5. When JefFerson saw the dangers of Euro- 
pean occupation of the Mississippi Valley, 
he subordinated' his preconceived views 

□ 

□ 

□ 

about the strict construction of the con- 




stitudon and joined with the majority of 
Congress to seize the opportunity to open 
the condnent to the expansion of thel 
United States, ' 

□ 

□ 

□ 

6. Jefferson changed his mind about the 




consdtudonality of the Louisiana Pur- 
chase. 

□ 

□ 

□ 

The profiles of this exercise are as follows: 
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ADVANCED— GROUP I 
EXERaSE VIII 


Item 1 Item 2 

— ' — - ' Gradei 

7 8 9 10 11 12 13 14 IS 16 17 789 10 11 12 13 14 IS 16 17 


99 


96 


93 


90 


87 


84 


81 81 .. 


78 


75 


74 

72 72 


— 71 71 

69 69 


66 TT 


65 6S 

63 ~ 


60 


57 .. S7 

57 57 

54 


SI 

51 

48 * 


45 45 

46 

42 


39 


36 

38 ~ 

. . — . . . . 36 



^cnsinccscorcy ■■ “■ 

30 J 

32 

“ ..30 




27 27 ^ 27 


Mean pet cent correct =6'97/l I 
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ADVANCED-GROUP I 
EXERCISE VIII 


Item 3 Item 4 

Grades * 

7 8 9 10 11 12 13 -14 15 16 17 789 10 11 12 13 14 IS 16 17 


99 




81 81 .. 


78 


75 75 


72 


69 


68 

66 = 67 


63 


62 

60 “ 


57 


54 .... 54 


— 52 

51 


50 

48 


45 


43 

42 “ 


39 


36 

.. 36 


“ 34 

— m ^9 

33 (chance score) 

30 

JJ Jj 

— 31 

..30 

27 ... 

27 

24 

24 24 .. 

21 

21 

18 


IS .. 


12 

13 


Mean per cent correct =699/11 =306/11 
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ADVANCE1>-GR0UP I 
EXERCISE VIII 


Item 5 Item 6 

— ■■■■ - Grade» 

7 8 9 10 n 12 13 14 IS 16 17 789 10 11 12 13 14 IS 16 17 


99 




87 


86 

84 


83 

81 81 


78 


75 


72 


69 


66 


65 6S 

63 ~ 


62 62 

60 “ 


59 

57 = 


56 

54 “ 


51 


50 

48 


45 


42 


40 

36 ~ 



7'f 

30 


27 

.. ..27 

24 


21 


18 

..18 

\ 


9 

8 

■ - 



•teMHean per cent correct *709/11 =336/11 
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ADVANCED EXERCISES 


classes have brought to light that many of the more thoughtful 
students responded on the basis of previous information about the 
situation pictured. The question arises then, “Is it possible that 
in the matter of drawing conclusions from evidence, the less one 
knows, the better?” It might be helpful before resetting this 
group of exercises to give the test to two groups of students of 
comparable intelligence, one group with history training and one 
group without it. Some such procedure should throw light upon 
the interpretation of the behavior of the more difficult items. 

The third problem to be taken into consideration concerns the 
amount of material that can be presented as evidence and yet be 
well-considered in the use of a number of items within a reasonable 
length of time. An obvious weakness in the exercises of Group I 
is that the constructor of the tests attempted to approximate the 
kind of problem which would be experienced in a “life situation.” 
That is, enough material was presented as evidence so that some 
selection would be necessary. Many high school students who 
were given the exercises complained that they could not do justice 
to Group I within the period. Teachers remarked, also, that the 
less careful students “finished” the exercises first. The suggestion 
of the constructor of the tests is that the evidence be reduced to 
the smallest amount necessary to the consideration of the items, 
and that a few items which require an impracticable amount of 
evidence be excluded. 

For example, in Exercise VIII, a brief statement, such as, “The 
Constitution does not state that the Congress or the President 
shall have power to purchase land, though some statesmen have 
believed this power may be implied from some general statements 
in the Constitution,” might replace the quotations from the Con- 
stitution. The last two sentences of the evidence and possibly one 
or two others could be omitted. 

A considerable space has been devoted to a discussion of the 
difficulties of Group I of the Advanced Test since this was the only 
group which showed such unpromising returns that Dr. Kelley 
considered it necessary to renew the entire attempt to build the 
exercises. 
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A more detailed description of the other exercises seems imprac* 
ticable here, but those exercises are available through the G)m- 
mission. The following statement summarizes the general condi- 
tions of this group of exercises. 

It has already been stated in this article that the two over- 
lapping exercises in Group I, that is, the last two of the Elementary 
Test and the identical exercises used as the first two in the Ad- 
vanced Test, proved to be too easy for college levels and, in some 
cases, too easy for high school levels. 

Of the 47 responses called for by the seven remaining exercises 
in Advanced Group I, 22 showed fair or good promise, S showed 
either a weakness in the structure of the Item or incorrect scoring, 
and the remaining 20 had insufficient discriminating power. 

Advanced Test 

Group II {Exercises in Making Proper Implications from Facts) 

Seven of the eight Exercises in Group II brought responses 
which made the exercises promising. All, except Exercise VI and 
Item 1 in Exercise VII, were recommended by Professor Kelley 
to be retained. 

Exercise V, with its profile, is here given as an example: 

EXERCISE V 

In 1861 a civil conflict broke out between the government of the United States 
and a group of its Southern States which had seceded and formed the Confederate 
States of America with Jefferson Davis as President. 

The following statements are quotations from English political leaders during 
that struggle: 

Given facts: 

I. The London Times of October 8, 1862, quotes Mr. Gladstone, who was 
Chancellor of the Exchequer, as saying, '‘Jefferson Davis had . . . made 
a nation. . . . We may anticipate with certainty the success of the Southern 
States." 

II. Mr. Mason, who was the commissioner from the Confederate States to 
England, wrote, *Tt is perfectly understood in the House of Commons 
that the war professedly waged to restore the Union is hopeless and the 
sympathies of four-fifths of its members are with the South." 

III. Mr. Belmont, an American who was an a^t for Rothschilds in England, 
says that he talked widi the English Pnme Minister, who said to him, 
"We do not like slavery, but we want cotton, and we dislike very much 

f our Morrill tariff.” 

n 1863, John Bright, the labor leader, at an immense meeting of trades- 
unions said: "PrivUege has shuddered at what might happen to old Europe 
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ADVANCED-<3R0UP II 
EXERCISE V 

If 


Item 1 Item 2 

Grade! 

7 8 9 10 11 12 13 14 15 16 17 789 10 11 12 13 14 IS 16 17 


99 

100 

96 

j 

93 93 

95 

90 


87 

88 

86 86 

84 “ 

84 

81 

83 82 = 

81 

80 

78 = 

80 

77 76 

75 


74 

72 

.. ..72 

69 


68 

66 ~ 

68 

65 

63 ..63 


60 


57 


54 .... 

55 

51 


48 


45 


44 

42 — 





Mean per cent correct >>812/11 *>888/11 
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ADVANCED-GROUP II 


EXERaSE V 






Itbm 3 






7 

8 

9 

10 11 

12 13 

14 

IS 

16 

17 

ito 


• , 

. 

4 4 4 4 

4 4 4 t 





109 




4 • 4 4 

.. 





• • 

• 

• ■ 


4 4 4 4 






106 

• • 

• • 








. ^ 

4 


' • 

4 4 






103 

• 



4 4 






102 

4 • 



♦ 4 4 4 






99 

• • 

• . 


4 4 4 4 






96 

• • 

4 « 

4 4 

4 4 4 4 






93 

• * 

. . 

4 4 

4 4 4 4 






90 

• • 

. . 

. . 

4 4 4 4 






87 

• • 


•• 

4 4 4 4 

86 


86 


86 


84 

•• 




82 83 ’ ' 





81 

•• 

• • 

• • 


• • = •• 





78 

■ ■ 

• • 

■ • 

.. 





77 

75 

• • 

• A 

• • 

4 4 4 4 






72 

t • 


• ■ 

70 






69 

• 4 

67 


.. 






66 

• • 



4 4 4 4 






63 

• * 

• » 


4 4 4 4 






60 

• 4 

■ 4 

* 4 

4 4 4 • 






57 

• 4 

« 4 

4 4 

4 4 * * 






54 

• 4 

52 

• 4 

4 4 

V 

4 4 4 4 






51 


•• 

•• 

4 4 4 • 







Item 3: Fur. Exerdse eodn is piominng. 
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if this grand experiment (of democracy In America) ahould succeed. But 
you, the workmen, you struggling upwards toward the light with slow and 
painful steps — ^vou have no cauM to look with jealousy upon a country 
which, menaced by the great nations of riie globe, is dtat one where labor 
has met with the highest honor.” 

Place a check in the appropriate square, or squares, after each of the following 
statements to show which given fact, or facts, show this statement to be true. 




Fact 

Fact 

Fact 

Fact 



I 

11 

III 

IV 

1. 

Some English leaders were interested 
in the Civil War for commercial 

□ 

□ 

□ 

□ 


reasons. 

2. 

Some labor leaders believe that the 






favorable attitude of the English 
leaders toward the South was due in 
part to their jealousy of the success 
of the American republic. 

□ 

□ 

□ 

□ 

3. 

English leaders seemed to think that 
the great military disadvantage 
accruing to the South because of the 






fact that they were in rebellion 
against a government already estab- 
lished and recognized was more than 
offset by various advantages. 

□ 

□ 

□ 

□ 


Exercise VI, which failed to show promise is also given here: 


EXERCISE VI 


Given facts: 

1. From 1821 to 1830 many people from the United States moved to Texas and 
were given land by Mexico. 

2. The Texans, who were largely from the United States, fought for and won 
their independence from Mexico in 1836. 

3. Texas asked to join the United States. 

4. Texas people said they must be allowed to keep their slaves. ^ J 

Place a check in the appropriate square, or squares, after each of the following 

statements to show which given fact, or facts, show this statement to be true. 
Statements: 


ft 


1. The request of Texas for admission 
to the United States presented this 

roblem: whether it would be wise to 
ave additional members of Congress 
who favored the extension of slavery. 

2. Texas was an independent nation 
during a part of its history. 

3. Mexico endangered the integrity of 

its own territory in its eagerness to 
settle its open lands. 


Fact Fact Fact Fact 


1 

2 

3 

4 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 
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ADVANCED--GROUP II 
EXERCISE VI 






ADVANCED EXERCISES 


ADVANCED-GROUP II 
EXERCISE VI 


Item 2 Item 3 

Grades 

7 8 9 10 11 12 13 14 IS 16 17 789 10 11 12 13 14 IS 16 17 


99 


96 


93 93 


90 


87 


85 86 86 

84 .... 84 = 

86 86 

82 — 83 82 

81 .. ~ 

o! 

00 

1 

OD 1 

80 

78 

79 

78 

75 

77 

72 


71 71 

69 — , . 


66 

68 

64 

63 ~ 

64 

60 


57 

57 

54 


51 


Mean per cent correct 
=903/11 

Item 2: Negligible differentiating 
power in this item. 

842/11 

Item 3: No differentiating power 
in this item. 


Since Exercise VI was not needed for Forms A and B of the 
revised test, no study for the purpose of discovering the cause of 
the failure of this exercise has been made. The organization of 
the group into Forms A and B is indicated in the summary. 
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Advanced Test 

Group III 

Of the 35 items in this group, 22 showed jair or good promise, 2 
items gave indication of being incorrectly scored. The other 11 
items showed too little differentiating power. 

Exercise V is given as an example here and the profiles of Items 
1, 2, 5 and 7, each of which illustrates a different type of behavior. 

EXERCISE V 

At the cIoK of the Revolutionary War by which the United States gained its 
independence, some of the people of the United States were better oflf financially 
than they had been at the beginning of the war, though the government was in a 
desperate condition financially. 

By checks placed in the appropriate squares, show which of the following state- 
ments would indicate improved financial conditions of certain groups, which 
would indicate government financial conditions to be in a desperate condition, 
and which statements you would reject as not showing either of these conditions. 


Item 1 shows differentiating power up to grade fifteen. Its failure 
to continue on levels sixteen and seventeen raises again the ques- 
tion of greater difficulty for more advanced students whose think- 
ing includes many other facts which suggest other possibilities 
than the one considered desirable here. 

Item 2 shows the reversal of expectation, which indicates im- 
proper scoring of answers. 

Item 5 presents the confused picture of an item with no dif- 
ferentiating power, and Item 7, the steady rise of a promising item. 

The two forms A and B have been set up, but it would certainly 
appear that further experimental work should be done to secure a 
better test of this type. Two suggestions are: (a) that the whole of 
Group III be submitted to 15 college professors for their criticisms 
of material, phrasing of items, and desirable responses, and (b) 
that the exercises be divided into two groups, one for grades 
seven to eleven, inclusive, and one for grades twelve to seventeen, 
inclusive, ns was suggested for Group 1. 
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Indicstes that 
aome were 
better off 
financially 

Indicates that 
government was 
in bad financial 
condition 

Does not 
show d^er 
condition 

1, During the war the fiihing 
industry and the shipping bust* 
ness were destroyed. 

□ 

□ 

□ 

2. The foreign armies brought with 
them the coin of their various 
countries. 

□ 

□ 

□ 

3. Congress issued a great quan- 
tity of paper money which 
depreciated in the hands of the 
people until it became valueless. 

□ 

□ 

□ 

4. When a foreign army was quar- 
tered for a long season in any 
place, the farmers in the neigh- 
borhood received better prices, 
and usually in specie, for their 
products. 

□ 

□ 

□ 

5. In June, 1783, a band of 80 
soldiers marched to Philadel- 
phia and demanded their pay 
at the point of the bayonet. 

□ 

□ 

□ 

6. In 1781 Congress proposed an 
impost duty of 5 per cent on 
some articles in order to raise 
money to pay the national debt, 
but not all of the states con- 
sented and the project fell 
through. 

D 

□ 

□ 

7. During the Revolution the 
country gained about 300,000 
inhabitants. 

□ 

□ 

□ 

8. While the British army was 
quartered in the cities of Bos- 
ton, New York, Philadelphia, 
they had spent money lavishly 
on entertainment. 

□ 

□ 

□ 

9. Congress had no power to col- 
lect taxes. 

□ 

□ 

□ 
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ADVANCED-GROUP III 
EXERCISE V 


Itui 1 Item 2 

< ■' ' ■' Gradet 

7 8 9 10 11 12 13 14 15 16 17 789 10 11 12 13 14 15 16 17 

99 




75 


72 72 .. .. 


69 


68 67 

66 65 ~ 


64 = 

63 


61 

60 


57 57 


54 


53 52 

SI 


48 


45 


42 


39 

39 

36 




30 30 .. 

27 

31 

— 29 

28 

24 


21 


Ig 

20 19 

18 

IS 


12 

14 U 

9 \ 


6 

8 

5 

Mean per cent correct »623/ll 

=225/11 
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ADVANCED— GROUP III 
EXERCISE V 


Item 5 Item 7 

, Grade»^ — — — 

7 8 9 10 11 12 13 14‘1S 16 17 7 8 9 10 11 12 13 14 IS 16 17 

99 

100 

96 95 

96 . 95 . 

94 

93 93 

91 “ 

90 90 

= 94 

93 

91 

87 


86 85 

84 = 

86 86 

83 

81 ~ 81 .. 


80 

78 “ 79 

75 

78 

76 

72 


69 


66 


63 


60 


57 


54 

% 

SI 


48 


45 


42 


39 


36 


H /.V* \ 


Mean per cent correct*9S7/ll «=851/11 
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Advanced 

Group /f' (To discriminate among types of evidence) 

Of the 30 responses called for by these exercises, 23 showed fair 
or good promise, 2 showed evidences of improper scoring, and 4 
showed slight differentiating power. Since these latter four were 
not needed in the revised test, no study has been made of their 
performance. 

Exercise II is given here with its profiles. 


EXERCISE II 


The following statement! concern President Johnson who succeeded President 
Lincoln. 



General Specific 
fact fact 

Writer’s 
personal 
opinion of 
situation 

Writer’s 
personal 
opinion of 
motive 

1-. Early in life he had been a poor 
tailor. 

□ 

□ 

□ 

□ 

2. His administration would have been 
more successful if he had possessed 
Lincoln's tact. 

□ 

□ 

□ 

□ 

3. Many of his recommendations proved 
to be very valuable to the nation. 

□ 

□ 

□ 

□ 

4. Johnson dtept most of Lincoln’s 
cabinet because he wished to follow 
Lincoln’s policies. 

□ 

□ 

□ 

□ 

5. When Johnson was impeached and 
tried by the Senate the vote for 
conviction was 35 ayes to 19 noes. 

□ 

□ 

□ 

□ 


The question arises whether the exercises in Group IV are in- 
trinsically better than ^those in Group I, or whether the brevity of 
the material permitted a more thoughtful handling of each item, 
thereby brining more consistent responses. 

A summary of riie entire Advanced Test follows; 
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ADVANCED-GROUP IV 
EXERCISE II 


Item 1 Item 2 

G rade* 

7 8 9 10 11 12 13 14 IS 16 17 789 10 11 12 13 14 IS 16 17 
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ADVANCED-GROUP IV 
EXERaSE II 


Item 3 Item 4 

- - Grades 

7 8 9 10 n 12 13 14 IS 16 17 789 10 11 12 13 14 15 16 17 


1 

99 

100 

96 


93 

94 

90 


87 


84 

84 

83 

81 


78 

79 

78 = 

76 ; 

75 

76 

72 72 


71 71 

69 = 

70 

66 


63 

64 

.... 63 

60 


57 S7 


55 

54 ~ 


52 52 

51 ~ 


SO 

48 


45 


42 42 V 


Mean per cent correct«681/ll 

Item 3: Good 

= 819/11 

Item 4; Good 
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ADVANCED-GROUP IV 
EXERCISE II 



99 


77 
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• 

• • 

• • 

• • 

• • 

• * 
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• • 

• • 
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• • 
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65 

64 

♦ • 
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• « 
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•• 

•• 

• • 


■ * 
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s 

51 



. . 

. . 

• - 
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• • 

• ■ 

• 
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. . 


• > 


• • 
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• 

45 
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• • 


• • 


• 

42 



• ■ 
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• 

* 

• 

0 0 

39 

38 
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•• 

• 

0 0 

0 0 
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• . 


•• 
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Mean per cent correct *828/11 Item 5: Good _ _ 

Sum of mean per cent correct *70.27 Exerciie entire is very prommnc 
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ADVANCED EXERCISES 
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Chapter V 


INTERESTS AND ATTITUDES 

The planning committee recognized among the expected out- 
comes of instruction in this field points of view, interests, and atti- 
tudes. Its statement was: 

Poinu of Fiew, Intertsu, and Aitituies: 

Point! of view, intereits, and attitudes are usually accepted as outcomes of the 
teachiM of these subjects. All propaganda rests upon that belief. Such attitudes 
are difficult enough to define, and still more difficult to test or measure. A promis- 
ing beginning has been made and it seems worth while to support further effort. 
Among the points of view, attitudes, and interests, arc the following: 

A perspective on current affairs. The task of affording pupils a basis for viewing 
events about them in a reflective manner it shared by nearly all of the social studies. 
This value arises more specifically from the comparisons with, or contrast to, similar 
activitiet in the past or in distant lands. Tests for this value must be constructed 
out of present situations. 

Historiealmindedness. Usually characterized as the habitual association of social 
occurrences wth the time at which they took place. It means, more fully, a recog- 
nition of society as a constantly changing complex, whose details vary as the com- 
plex changes. 

Loeationaltnindedness, The habitual association of social occurrence with the 
place in which they occurred. 

Concern for the common good. Patriotism is a more common designation of this 
objective. It involves training the individual to recognize his own welfare in the 
welfare of the group, or to respect the common welfare as a desirable end in itself. 

Tolerance, racial, religious, national, social. Related closely to the preceding, it 
deals.more specifically with certain prejudices which seem inherent and lead to in- 
justice. It is believed that acquaintance with and understanding of other races, 
religions, parties, and social groups will lead to a fairer judgment of them, with 
resultant oeneflt to the individual and Ktciety. 

Leisure time 'inters As. Acquaintance with the social world, past and present, is 
commonly regarded as capable of arousing in the pupil a passion or interest to 
find out as much as possible about some congenial topic, or groups of topics, whose 
pursuit will afford him satisfaction and pleasantly occupy his leisure. With in- 
created leisure a common condition, this objective becomes increasingly important. 
Tests must be devised. 

The adviser on tests felt a special interest in some of the more 
fundamental aspects of the problem of attitudes. With M. R- 
Trabue, he undertook to contrive a test of certain personality 
traits, vdikh is here reported. He also engaged A. M. Jordan to 
make a supplementary study of the relation of pupils' cheating on 
examinations to certain circumstances. This study is also reported 
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here.* The progress of attitude testing was so far advanced that it 
seemed best to engage L. L. Thurstone to construct tests of atti- 
tudes on certain questions. The Commission selected a series of 
issues for which Thurstone constructed tests. His technique, which 
had worked so well at adult levels, presented some difficulties when 
applied at the earlier grades of school life. These studies are dis- 
cussed elsewhere in this volume.* 

>See page 437 ff. 

’See page 83 fF. 
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THE MEASUREMENT OF CERTAIN TRAITS 
OF CHARACTER 

Truman L. Kelley and M. R. Trabue 
Section I 

Historical Statement 

The study of character has been and still is being attempted by 
techniques which, because of their lack of precision, were discarded 
long ago in such other psychological fields as psycho-physics, 
mental measurement, vocational abilities, scholastic achievement, 
and the like. The development of objective measures of character 
traits is a fundamental necessity in the improvement of methods 
of character training and in the remedial work connected with 
penology, delinquency, and other manifestations of maladjust- 
ment. Serviceable tools for measuring character status objectively 
must be developed before it will be possible to demonstrate con- 
vincingly the superiority of any method of character training or 
correction. 

The analysis of Spranger and the methods of measurement 
developed by Strong, Allport, Bemreuter, Thurstone, and others 
give promise of instruments of measurement and analysis suffi- 
ciently radical, novel, and reliable to suggest the opening of a new 
field of psychology. As this tends to be accomplished, it may be 
expected that one attendant source of error in character measure- 
ment will grow larger and larger, — that connected with disin- 
genuous responses. The temptation to give responses which 
indicate a socially approved character rather than those which in- 
dicate the true character of the subject will become increasingly 
powerful as more and more practical dependence is placed upon 
the results of these measurements. 

Coaching is serious enough in connection with the measurement 
of achievement. Though a child could not take a Thorndike 
psychological examination and earn a college entrance score with- 
out actually knowing a good deal, by a study of the Thurstone 
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scoring key he might learn the '‘right" responses in an ^attitude 
scale, thereby making a higher mark, even though his character 
had been negligibly improved, or possibly changed in the opposite 
direction. The vocation interest scores resulting from taking the 
Strong Interest Blank are less likely to be distorted by falsification 
of the record because ,(2) a falsification characteristically will lower 
the interest score in one vocation while increasing it in another, 
which being of questionable advantage to the one taking the 
test constitutes a very weak stimulus to prevarication, and be- 
cause (b) it is frequently difficult or impossible for the taker of the 
test to know whether a given falsification will have a desired result 
upon his score. 

For quite similar reasons the authors believe that insincere 
responses will not seriously affect character indexes obtained from 
a free association test, wherein a stimulus word is presented and 
the subject responds with the first word that comes to mind. It 
is true that a subject, knowing that an “honesty” score is to be 
calculated, might intentionally alter all of his responses with intent 
to get a high score. However, since the association test as here 
developed and scored by the authors results in eight character 
measurements, rather than in one, and since a modification of 
responses to raise one character score would typically tend to 
alter, frequently to lower, other character scores, and since the 
subject is scarcely wise enough to know what is going to happen 
to his scores as a result of giving disingenuous responses, — even 
the authors cannot tell him except in a very few instances, — there 
seems to be little possibility of a systematically successful falsifi- 
cation of response on the free association test. In fact, the authors 
consider it the most nearly coach-proof test of all the psychological 
tests which involve conscious intelligent responses. 

If it is possible to give a free association test, score it several 
ways for several character traits, and obtain reliable trait measures, 
a tool is made available for the study of a socially important and 
far-reaching type of phenomenon. The authors do not claim as 
yet to have accomplished this result, but the validity and reliability 
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of the trait measures thus far secured give promise that the prob- 
lem is soluble in a practical sense. 

In skeleton outline, the study may be divided into certain steps 
upon a chronological basis: the authors, familiar with the attempts 
at character measurement and the uses of the free association 
experiment in the study of abnormal psychoses and of normal 
interests, considered it a promising instrument to adapt to the 
measurement of character. The steps in the process appeared to 
be as follows: 

A. The selection of an appropriate list of stimulus words. 

B. The giving of these stimulus words one at a time to sub- 
jects, securing first responses that come to mind, thus 
catching native or deep-rooted tendencies to associate. 

C. The securing for these same subjects of trait ratings 
based upon judgments of associates, fellow students, and 
teachers, which ratings constitute entirely independent 
measurements of the traits it is desired to measure by 
means of the test. 

D. A weighting or attachment of appropriate credit to each 
response given by these subjects for each of the traits 
rated, thus providing a scoring scheme or means of grad- 
ing the responses of a new subject to the same stimulus 
words. 

E. A checking of the validity and reliability of the instru- 
ment, including its attached scoring scheme, by com- 
parison of the trait scores earned by the members of a 
new group with entirely independent trait ratings. 

The steps just mentioned were undertaken and completed under 
the auspices of the Commission on Investigation of the Social 
Studies in the Schools. They resulted in an instrument with 
reliability for each of eight character traits in the neighborhood 
of .35. This reliability being inadequate for individual measure- 
ment, the authors were aided by a grant from the Carnegie 
Foundation for the Advancement of Teaching given to Harvard 
University by means of which they have been able to investigate 
the adequacy of the instrument as a means of measuring the mean 
standing of class groups in the same eight traits. 
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This Study has revealed certain weaknesses in procedure which 
should be remedied in the future, and, in the opinion of the 
authors, confirms their earlier judgment that the instrument is a 
valid one for the study of character traits in groups. There remains 
the essential process of refining and modifying it still further, so 
as to make it a trustworthy instrument for the measurement of 
individuals. Suggestions to this end will be made later. 

Section II 

Selection of Character Traits and Criterion Measures 

Critical examination of numerous discussions of character, 
morals, ideals, citizenship, and the like indicated that writers 
have frequently attached very different meanings to the same 
trait name, as well as different trait names to the same type of 
social behavior. Statistical tabulations showing the relative fre- 
quency with which each trait appears in the literature were there- 
fore not considered entirely appropriate as final determinants of 
the characteristics to be employed in this experiment with the 
free association test. Additional criteria were accordingly adopted 
and used in reducing the number of traits to eight, which was 
thought to be as large a number as could be handled in such a study. 

In the first place, the effort was made to select only those traits 
which are likely to have real and immediate significance in making 
adjustments to social institutions and to one’s fellows. Such 
characteristics as an attitude of emotional detachment to the con- 
flicts reported in history may be of great value to a young scholar 
in the library, but the connection between this attitude and the 
making of happy adjustments with his fellows and with his im- 
mediate social environment is not so clear and certain. Unless the 
connection between the presence of the trait and the development 
of desirable personal and social relationships was fairly obvious, a 
trait was not included in this study. 

A second standard used in the selection of characteristics was 
the degree to which a trait differed from the other traits being con- 
sidered. Persistence, for example, is an often-recommended char- 
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acter trait, but it is closely related to the trait called '‘school 
drive," which had been chosen for other reasons. While the inter- 
correlations of the measures developed in this study indicate that 
there are common elements in the traits selected, a serious effort 
was made to choose for investigation traits that seemed to be as 
distinct or discrete as possible. 

A third criterion of selection grew out of the method to be used 
for obtaining independent measures of the traits selected. Since 
each pupil was to be judged by his fellows and by his teachers as 
to the degree to which he possessed each of the eight traits, it was 
desirable to select only those traits in which evidences of presence 
or absence could be readily observed and accurately estimated by 
teachers and pupils. It would have been wasteful of effort to have 
included a trait in which ratings by those who knew him best 
would be unreliable or lacking in validity. 

Closely related to this was the fourth criterion, that only those 
traits should be chosen for experiment in which other useful tech- 
niques of measurement had already been developed or might, in 
the judgment of the authors, soon be developed. While the 
practice of validating one test by its correlations with another has 
definite limitations, it is certainly desirable in the development 
of any new testing technique to be able to check the results of 
the new method against the results obtained from independent 
measures of the same traits. 

While the traits selected by the application of these criteria may 
not be the most important elements of social character, they were 
as good a selection as the authors could make for their study at 
the time the choices had to be made. It was assumed that, if the 
free association test technique could be made to reveal the relative 
strength or weakness of each of these eight traits, it would be 
possible later to develop scoring keys for revealing the strength 
of any other equally identifiable trait or attitude. 

The eight traits chosen for the experiment are indicated here- 
with, each with brief cl^aracterizations of both its positive and its 
negative behavior patterns. 
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CX)URTESY 


{Poritioe) 

Considerate, courteous, polite, gra* 
cious, diplomatic, refined, well-bred, 
tactful, gentlemanly (ladylike), chiv- 
alrous, urbane. 


{Negativt) 

Inconsiderate, discourteous, impolite, 
rude, unmannerly, unaiplomatic, 
crude, ill-bred, tactless, ungentle- 
manly (unladylike), barbarous, boot- 
ish, rustic. 


FAIR PLAY IN PLAYGROUND GAMES AND CONTESTS 


{Positivf) 

Always plays by the rules, even when 
it means that his side may lose. 

Insists on applying the same {ules in 
the same way to both sides. 

Never tries to cheat or to take an 
unfair advantage of the other side. 


HONESTY IN 

(Positive) 

Does not copy materials from other 
children or from books. 

Does not cheat on tests and examina- 
tions. 

Reports practice-test scores correctly. 

Does all his own work exactly as he 
was told to do it. 


(Negative) 

Docs not play fair or follow the rules 
except when they favor his side. 

Tries to apply the rules in one way 
for his own side and in another way 
for the other side. 

Takes unfair advantage of the other 
side whenever he has a chance. 


SCHOOL WORK 

(Negative) 

Hands in, as his own, materials copied 
from books and from other pupils. 
Cheats on tests and examinations. 
Lies about his scores on tests. 

Gets help in doing assignments that 
were to be done oy himself. 


LOYALTY TO HIS FELLOWS 


(Positive) 

Is always loyal to his chosen leaders 
and playmates. 

Will sacrihce his own pleasure for the 
good of his gang. 

Never “tells” on a member of his 
gang. 

Stands by his friends “through thick 
and thin.” 


(Negative) 

Is not loyal to his leaders and play- 
mates. 

Will sacrifice his gang for his own 
pleasure. 

"Tells” on his playmates if he thinks 
it will help him. 

Deserts his friends when they are in 
trouble. 


MASTERY 


(Positive) 

Masterful, rommanding, dominant, 
authoritative, domineering, active, 
rough, forceful. 


(Negathe) 

Submissive, obedient, compliant, de- 
ferential, yielding, easily led, pass- 
ive, gende, mild. 
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POISE 


(Positive) 

Compoied, calm, confident, poiied, 
•elfi^ontrolled, emotionally stable, 
well-balanced, judicious, prudent, 
discreet, consistent, careful, deliber- 
ate, thoughtful. 


(Negative) 

Nervous, anxious, agitated, flighty, 
easily-excited, emotionally unstable, 
eccentric, injudicious, imprudent, 
indiscreet, inconsistent, reckless 
hasty, thoughtless. 


REGARD FOR PROPERTY RIGHTS 


(Positive) 

Always recognizes an owner’s right to 
control and to protect his own 
property. 

Asks permission before he borrows 
anything; returns things promptly 
and in good condition. 

Avoids injury to property of others, 
but tries to make good any injury 
done accidentally. 

Tries hard to find the owner in order 
to return anything he has found. 


(Negative) 

Takes things he wants, regardless of 
who owns them. 

Forgets to return and does not take 
care of things he has borrowed. 

Deliberately defaces and injures the 
property of the school and of others. 

Runs away and denies responsibility 
when he has injured someone's 
property. 

Keeps anything he finds, even when 
he knows to whom it belongs. 


SCHOOL DRIVE 


(Positive) 

Tries to learn all his lessons perfectly. 

Gets to work immediately without 
any complaints. 

Does not give up when a lesson is long 
and hard. 

Tries to do well everything the teacher 
asks him to do. 


(Negative) 

Never does an assignment well if he 
can get out of it. 

Complains and objects to assignments. 

Gives up trying to get a lesson as soon 
as he can find an excuse. 

Frequently prevents others from doing 
good work. 


The above words and phrases descriptive of each trait and its 
opposite were used in obtaining ratings by teachers and super- 
visors of the pupils in each class examined. Each teacher was 
asked to indfcate for each trait those pupils in the class who were 
outstandingly high or low in the quality. The sheets on which 
these different ratings of a class were obtained were indentical 
except for the titles and) descriptions of the traits. Facing this 
page is a reproduction of the sheet on which these 

ratings were made. The mimeographed instructions to the teacher 
making the ratings were as follows; 

DIRECTIONS FOR RATINGS OF PUPILS BY TEACHERS 

1. Do not attempt to consider more than one trait at a time. In connection with 
each of the eight characteristics, take at least two days in which to observe 
closdy the behavior of the pupils. 
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la alwaya loyal to Ua ehaaan laadara and playmataa. 
Will aacridea hie own pleatura for the g^ of Ua 

gang. 

Ravac '‘talla” on a member of Ua gang. 

Staada by Ua frlenda “throngh thick and tUn.” 





PIamb lift ftt tha top of this oolama th« raxM of tho pqpll who 
hu tko (reotMt UROut 0/ th« pooitivo trait. Lift BMoad thi 
itai of th« popll who hu tho aoxt'lArfOfi omovot, tod m oa. 



(A nndacate mount of the poutiTe trait) 










Scboot 


Gn4i 


>•«»»«*%, 


WHO IS IT? 


■4 

Every citu Ml im E e fern Myt mi fMi wM t» teme wty ar otMi an Ml HM Ike atken. Tkb ii • 
tesi te see hem meU Uese m s ml fefib ere hiewm te their clmseteetu. 

Yets meed met ilf)i >««r hmm le Mi test mmUit yem wish teieee. YemrmmemmimUhemieihieilalltlkd 
itmdy mUch It M*t miedi at e msd t eit Uy end mU froheUy meeer he teem hj year t e ee i er or primeipel. Yem 
muy msHti year am aemu after aay amadiom le tthirh yem thimi U is the test mmtmer. If yem thimh me miemiher 
ef year dais It lihe the tenoa deterietd la a qmestiem, ship that gmttHem amd fe am le the meed. De met lerfet 
So mmEe both the frit mid last mamm ef may dassmsates mhe are lihe tM fersemi la the fattHaat. 


1. Wbo is it that always staadi up for his (or her) companions, and never tdk on a Meodf 


2. Wbo never gets exdtM, but is always calm and careful? 


i. Who is it that always triet to boas everyone else and to run everything? 


4. Who never tries to cheat or to take an unfidr advantage of the other aide when playing gaUMs? 


S. Who is it that doesn’t even try to get the lesaotH your teacher assigns, tiniest made to do It? 


6. Wbo is most often rude and impolite to people? 


7. Who is moat uHvoua and geU exdted most eadly? 


8. Who la it that taka other people's things without penniarion and tries to keep them? 


(***^) 




*, Who It tlimf* poHu ood ecmUtttU d stWr p«apit% fwlhni? 


10. Wbo It it that mw dtalt or dott uytUai that it ualair when Ukiag mmin t ti ota or I ttio flt ? 


11. Who It tlmjrt miy to lo toywlMte or do anythiat that anyone nicgeiti/ 


II. IVho It it that teOi oo playmatet, and iorgett Irieodi adten. they are io trouble? 


13. Who alwayi triet to find the owner in order to return anything that hat been lound? 


14. Who it It that doet not play aocordiog to the rulei and often takes an unfair advantage of the other dde? 


13, Who triei to do pwfectly everything the teacher aaks or ataignt? 


16, Who marti on .the waits and braika or injures pn^ierty? 


17. Who gets BOmeone ebe to do his (or her) school work or copies ansmrs instead o( woikiiq them out' alone? 


It. .Who always asks permltaion before borrowing anything, and always returns thinip ptompQy and In good 

cimtttiQn? 


YOU NEED NOt SIGN YOUR NAME TO THIS TEST. 









CHARACTER TRAITS AND CRITERION MEASURES 


2. Have a complete alphabetical list of the names of members of the class before 

you in maKing each rating. 

3. Rate each pupil as accurately as possible for the trait described on the blank. 

Do not allow your general opinion of the pupil or your recorded judgment 
regarding him in some othe* characteristic to influence you. 

4. It is not necessary to write on the blank the names of those pupils who have 

only an average or indifferent amount of the characteristic named. Record 
only the names of that third of the class having the greatest degree of the 
positive trait and the names of that third having the greatest degree of the 
negative trait. The names of those pupils who have the largest amount of 
the positive characteristic should be entered at the top of the column on the 
left, and the names of those having the largest amount of the negative trait 
at the bottom of the right-hand column. 

5. As soon as you have rated a given characteristic, put away your record sheet 

and give your full attention to the next characteristic to be rated. Do not 
allow your record in one trait to influence your judgment in rating another 
trait. 


Order of Ratings 

1. Courtesy 

2. Loyalty to His Fellows 

3. Poise 

4. Fair Play in Playground Games 

5. School Drive 

6. Regard for Property Rights 

7. Honesty in School Work 

8. Mastery 

Two teachers who knew the members of the class well, usually 
their regular home-room teacher for the current year and their 
regular teacher during the previous year, made out independent 
rating sheets for each of the eight traits. Occasionally the second 
rating of a class was made by the principal, the general supervisor, 
or the teacher of some special subject. Having two independent 
ratings of the members of each class made possible the calculation 
of the reliability of these ratings. (See Tables VI and IX, pages 
412 , 413 .) 

Still another independent series of ratings of the members of 
each class on approximately the same eight traits was obtained 
by having each pupil fill out anonymously a “Who Is It?“ blank. 
This blank and the directions for administering it are here repro- 
duced. 
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WHO IS IT? 

Directions for Administering 

The chief point* to be lought for in administering this test are (1) to secure 
independence of judgment from each pupil, and (2) to prevent pupils from being 
influenced by other considerations than the actual behavior of their classmates. 
Independence of Judgment can be obtained by insisting that there shall be no 
communication or looking at what anyone else has written. Honesty of judgment 
is more difficult to obtain, because many of the pupils who have the keenest jud^ 
raent have also the strongest sense of loyalty to their fellows. In case any hesi- 
tancy upon the part of the class is shown in writing names, an examiner should 
make it perfectly clear to the pupils that neither he, the teacher, nor the principal 
will look at the answers to this test, and that no member of the class will gain any 
advantage or disadvantage from the answers that arc written to these questions. 

See that each pupil is provided with a pen or pencil, and with a desk or table 
on which he can write comfortably. Hold up a copy of the test and make the 
following explanations: 

“This exercise is different from the usual school exercise. You need not sign 
your names to your papers, but you are to give the best answers you can to each 
question. Do not read the questions until I tell you to do so, but fill in at once 
the name of the school, your class or grade, and the teacher’s name.” Distribute 
the papers, one to each pupil, with the title page up, and then read aloud the 
directions printed on the test blank: 


“Every class has in it a few boys and girls who in some way or 
other are not just like the others. This is a test to see how well 
these unusual pupils are known to their classmates. 

"You need not sign your name to this test unless you wish to 
do so. Your answers will be used in a statistical study which is 
being made at a university and will probably never be seen by your 
teacher or principal. You may write your own name after any 
question to which you think it is the best answer. If you think no 
member of your class is like the person described in a question, 
skip that question and go on to the next. Do not forget to write 
both the fir^t and last names of any classmates who are like the 
persons in the questions. 


"Write the names of two of your classmates after each question, if there are 
two who are like the person mentioned in the question. No pupil wilt gain any 
advantage or disadvantage because you write his name. I shall have you put 
your paper into this large envelope yourself,” (hold up the container) “and then 
I shall send it away to a man who does not know you or your classmates. He will 
not report what you have said. Please be perfectly honest. Writing something 
that is not true would not help or hurt any of your classmates, but it might spoil 
a very important investigation which is being made at the university, and it 
would certainly not do you or anyone else any good. Answer each question as 
wdl as you can. Are there any questions?" 
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In aniwering quetdont, ute vh«r« possible the language of the instructions 
already given. Allow the pupils as much time at necessary to answer the questions. 

In taking up the papers, have the child himself insert hit paper in the envelope 
or do it yourself in his presence. Do not in the presence of the children take from 
the envelope any papers which '4he children have written, and do not allow the 
teacher to do so. If the children are to give us their honest judgments they must 
feel that they are being fairiy treated. 

By combining the ratings of the two teachers with the returns 
given by the pupils themselves on the “Who Is It?“ blank, all 
members of each class were identified for each of the eight charac> 
teristics. The weight to be assigned any particular response to a 
stimulus word in the free association test was later determined 
statistically, as explained in the next section of this report, by 
considering the difference in the frequency with which that re- 
sponse appeared among the pupils who had been rated high and 
among the pupils who had been rated low in the trait. 

The schools and classes in which these tests were administered 
were selected in terms of the following standards*. — (1) The pupils 
must be normal representatives of the total school population who 
have not had any special training in social attitudes and who do 
not present any highly unusual behavior problems. (2) There 
must be at least two teachers or supervisors who know the pupils 
well enough to give useful ratings of their behavior patterns and 
attitudes. (3) There must be at least one teacher or supervisor 
who understands the importance of uniformity in the adminis- 
tration of the tests and who will take responsibility for seeing that 
they are given according to the directions. (4) City, village, and 
country schools should all be included in order to avoid the in- 
fluence of any special type of environment. The village and 
country schools used were in North Carolina, while the city schools 
were in Washington, D. C. 

Administration of Word Association Test 

Previous investigators had found that many persons, when al- 
lowed to write response words without supervision or control, 
tended to develop stereotyped modes of reaction. The character 
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of the examples used in explaining the test had been recognized as 
influential in determining the type of reaction adopted. lf» for 
example, the response word given in an illustration was an oppo- 
site of the stimulus word, there was thereafter a tendency among 
those being examined to respond to each stimulus word by writing 
its opposite. Similarly, an example in which the response word was 
a part or specific instance of the thing named by the stimulus word 
tended to fix the whole-part relationship as the pattern to be 
followed in subsequent responses. 

Several devices were tried by the present authors in an effort 
to prevent the development of such uniform types of response, 
which could obviously reveal little regarding the subject's personal 
attitudes or individual character. It was thought that it would 
probably be advantageous to use in the directions a number of 
different illustrations, each representing a distinct type of asso- 
ciation, but all being of the character that might have significance. 
Among the responses to be avoided as types, though not as occa- 
sional indicators of associative process, were the “clang" reaction, 
in which the response is a word that sounds like or rhymes with the 
stimulus word, the “opposite” reaction, and the “whole-part” 
response mentioned above, the response which consists in giving a 
synonym or definition of the stimulus word, and the “predicate” 
response, in which one indicates something which is done by or 
with the thing for which the stimulus word stands. 

A further attempt to prevent the development of such stereo- 
typed responses was made by providing printed lists of response 
words in which the pupil being examined was to check the first 
response that had come to his mind. It was also hoped that the 
provision of these check lists might possibly reduce the amount of 
time required to secure responses and thus make possible an in- 
creased number of stimulus words in a single testing period. The 
printed lists were made up of those responses which were known 
to be most frequently given to each stimulus word, except that 
“opposites,” “clang” responses, and the like, were not always 
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included. Any child who actually thought of one of these stereo- 
typed responses which had been omitted would have to write out 
his response word. The purpose was not to rule out any type of 
reaction, but to encourage the more distinctly individual responses. 

Experimentation with mimeographed and printed pads, on each 
page of which one stimulus word and twenty appropriate response 
words appeared, indicated that many pupils were inclined to check 
the response words appearing at the tops of the two columns 
rather than those at lower points on the page. In other words, 
pupils were marking the first response that came to their attention 
from the printed page rather than the response which came from 
personal modes of mental reaction. To prevent their continuance 
in this practice, pupils were asked to enter in a half-inch square 
at the bottom of each page the first letter in the first word of which 
they had thought when the “key-word,” i. e. the stimulus word, 
for the next page was pronounced. After this initial letter had 
been recorded, they were asked to turn the page and check or 
write the entire word. Even this procedure failed to prevent a few 
pupils from depending on the printed list for suggestions, but the 
number of such pupils was greatly reduced by the device. 

In working on the administration of the association test, one of 
the important considerations in the minds of the authors was to 
obtain free responses to as many different words as possible, in 
order to secure maximum reliability in the resulting indexes to the 
individuars habitual thought patterns. With twenty response 
words printed on each page, many pupils in each class still found 
it necessary to write their responses. There was some evidence 
that twenty response words printed alphabetically in two columns 
required an undue amount of time on the part of certain pupils in 
the mere process of determining whether their own responses were 
among the twenty. The number of printed response words was 
therefore reduced to fourteen on each page, the stimulus word was 
omitted from the page, and the organization of the test pad or 
book was simplified. Each of the 200 pages following the first 
three, which were explanatory examples, was similar to page 49, 
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which IS here reproduced, and for which the stimulus word was 
Silk. 


.cloth 



cotton 



.dress 

Japan 


fabric 


thread 


□ 

49 

Stimulus words were selected from those previously used by 
other investigators, in order that those which frequently stimulate 
a single response, a narrow range of responses, or no response might 
be omitted, and in order to be able to use the findings of these 
investigators in choosing the most frequent responses to be printed 
on the test page devoted to each stimulus word. The Woodrow- 
Lowell* tabulations of children’s responses to the Kent-RosanofF 
words, Johnson O’Connor’s® tabulations of adult responses to the 
same words, Jennie B. Wyman’s* tabulations of children’s re- 
sponses, and P. £. Schellen berg’s* tabulations of the responses of 
five hundred college students were the chief sources from which 
the fourteen response words to each of the two hundred stimulus 
words were selected. An additional hundred stimulus words, each 
with twenty of its most common responses, were chosen at the 
same time, buMhese were not used in this study on account of the 
great length of time that would be required to secure responses to 
each of three hundred stimulus words. Approximately an hour and 
three quarters was required to secure responses from a junior- 
high-school class to two hundred words when given by means of 
the printed test books according to the following instructions: 

‘Woodrow, Herbert, and Lowell, Frances. Children’s Association Frequency 
Tables. Lancaster, P. The Psychological Review Company, 1916. 

•O^Connor, Johnson. Bom That IFay. Baltimore, Williams and Wilkins 
Co., 1928. ' 

*0n the Influence of Intncst on Relative Success, A Dissertation on file in the 
library at Stanford University. 

*A Dissertation on file in the library at the University of Minnesota. 
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NEXT-WORD EXERCISES 
Instructions for Administering 

See that each pupil is comfortably seated at a desk or writing table and is 
provided with a pen or pencil ^hich is in good working condition. Hold up a copy 
of the booklet and tell the pupils that you are going to give them an unusual but 
very interesting “word exercise.” Direct them to fill in at once their names and the 
other facts called for on the front cover of the booklet. Ask them not to open the 
hooks or to look inside until they are told to do so. Distribute the booklets, face 
up, and see that the names and other data are entered quickly and correctly* 

As soon as the blanks on the cover page have been filled, read slowly and 
distinctly the following explanation: — ■ 

“This is an exercise in which you are to tell the next word you think of after 
each word I pronounce. I shall say a word, and then you are to think of some 
other word which it suggests to you. Any word will do, if it is the first word that 
comes to your mind after I pronounce my word. The first letter of the word you 
think of is to be written in a little square before you turn the page, and then the 
entire word is to be written on a dotted line after you turn the page. 

"Now we are ready to try a word. As soon as you have thought of your word 
and written its first letter in the little square, hold up your hand. Never turn a 
page until you are told to do so. 

“Do you all see square No. 1, near the bottom of the page?" Pause until 
everyone has found it. In the lower elementary grades it will be helpful to suggest 
that each child put his finger on the number just below the square. 

“In square No. 1 write the first letter of the word you think of when I say 
the word ‘LAMP’.” Pronounce it distinctly and then spell it out, “L-A-M-P. 
Write the first letter of the word you thought of, and then hold up your hand." 
Pause until all but two or three hands are up. Observe which pupils fail to get 
their hands up, so that extra help may be given them after the other pupils have 
turned the page. 

“Now turn to page 2. Write on the dotted line the word you thought of. 
Hold up your hand as soon as you have written your word.” Pause until 90% of 
the hands have been raised, and then ask» “Do all of you understand what to do 
when I tell you the next word? First, you think of another word. Second, you 
write the first letter of your word in the square. Third, when I tell you to turn 
the page, you turn to the next page and write all of your word on the dotted line. 
Are there any questions?" Answer briefly any questions that may be asked. 

“Do all of you see square No. 2? Write in square No. 2 the first letter of the 
first word you think of when I say the word ‘RICH, R-I-C-H.’ Hold up your 
hand as soon as you have done it.” Pause until about 90% of the hands are up, 
and then say, “Turn to page 3 and write the word you thought of. Hold up your 
hand as soon as you have written it.” Pause until all but one or two hands are up. 

“Now look at square No* 3. Write in it the first letter of the word you think 
of when I say the word ‘RIDE.’ Then hold up your hand." Pause. "Now 
turn the page and write the whole word on the dotted line.” Pause until til but 
two or three have finished. 
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“Now read to yourtelvei, wh9e I read aloud what it sayi juit above square 
No. 4. X)n each of the next pages you wUI find 14 words, printed alphabetically 
in two columns. On each page, if the word you thought of is in the list, put a 
check mark in front of it. If the word you thought of is not in the list, write it on 
the dotted line." Pause. 

“Do all of you know how to look for a word in an alphabetical list? Suppose 
the word you think of begins with the letter B. Where would you look for it in an 
alphabetical list?” Pause for an answer. 

“Yes, that’s right! You would look among the words that begin with B, 
near the top of the first column. Now suppose the word you think of begins with 
the letter Y. Where would you expect to find it?” Pause for replies. “Yes. 
You would look among the words beginning with Y, near the end of the second list. 
When I tell you to turn a page, do not look at all the printed words. Look only 
at the words which begin with the same letter as your word. If the word you 
thought of is not where it ought to be in the list, it is not there at all, and you will 
have to write it, as you have written the first three words. If you find your word 
in the printed list, make a little check mark in front of it, instead of writing it. 

"Now look at square No. 4. In square No. 4, write the first letter of the word 
you think of when I say the word, ’AGE.’ Then hold up your hand.” Pause. 
"Now turn to page 5 and see if the word you thought of is in the printed list. If 
so, make a check mark in front of it. If it is not there, write it on the dotted line.” 
Pause. "Does anyone wish to ask a question?” Answer any questions. Be 
sure chat the pupils understand the procedure, and then continue as rapidly at 
possible, using the following directions for the next half dozen pages: 

"In square No. . . . write the first letter of the word you think of when I say 

the word " Pause. “Now turn to page . . . and check your word, or 

write it.” As soon as the routine is established, abbreviate the directions to the 
following; 

‘The word for square No. ... is ’’ Pause. “Turn to page 

*’ Do not neglect to give the number of the page and square each 

time, because it is very easy for the pupil to turn more than one page, especially 
when the last pages have been reached. 


Two or more class periods will be required for completing the test. Do not 
tty to give the entire test at one sitting. Allow at least one rest period during the 
test. 

When the initial letter has been written in square No. 101, have the pupils 
turn their books upside down to find page 102. See that everyone has found the 
place before pronouncing further words. 

STIMULUS WORDS FOR NEXT-WORD EXERCISES 

In pronouncing these qmrds, bear in mind that the word listed for page 5 
is pronounced while the pupils are still looking at page 4; that the pupils write 
the first letter of their response word in the square on page 4; and that they check 
or write their entire response word on page 5. 
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Page 

No. word 


1 


2 

lamp 

3 

rich 

4 

ride 

5 

age 

6 

dress 

7 

stranger 

8 

sundown 

9 

night 

10 

ocean 

11 

school 

12 

music 

13 

needle 

14 

restless 

IS 

river 

16 

poor 

17 

prison 

18 

rain 

19 

range 

20 

mother 

21 

name 

22 

church 

23 

citizen 

24 

pleasure 

25 

prefer 

26 

square 

27 

stage 

28 

loud 

29 

marble 

30 

command 

31 

COW 

32 

stove 

33 

summer 

34 

listen 

35 

luck 

36 

boy 

37 

captain 

38 

simple 

39 

sleep 

40 

blue 

41 

butterfly 

42 

trip 

43 

useful 

44 

handle 

45 

hay 

45 

cheese 

47 

city 

48 

rush 

49 

silk 

so 

watch 
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Page 

No. 


word 


Page 

No. 


word 


51 whistle 

52 cottage 

53 eagle' 

54 hurt 

55 kitchen 

56 pain 

57 platform 

58 fight 

59 lion 

60 crowd 

61 dance 

62 tire 

63 train 

64 friendly 

65 habit 

66 smooth 

67 soldier 

68 king 

69 learn 

70 flag 

71 form 

72 join 

73 joy 

74 fair 

75 f^ault 

76 country 

77 deep 

78 snake 

79 star 

80 queer 

81 college 

82 control 

83 fond 

84 honor 

85 table 

86 easy 

87 enjoy 

88 master 

89 moon 

90 new 

91 rose 

92 aim 

93 bat 

94 sheep 

95 short 

96 delighted 

97 diamond 

98 narrow 

99 old 

100 walk 

101 young 


102 wait 

103 wild 

104 over 

105 note 

106 difficult 

107 desire 

108 sickness 

109 shock 

110 black 

111 band 

112 seed 

113 once 

114 mountain 

115 milk 

116 evening 

117 eating 

118 thief 

119 swift 

120 home 

121 admire 

122 companion 

123 cave 

124 pure 

125 sorrow 

126 sing 

127 dark 

128 current 

129 fairy 

130 exercise 

131 journey 

132 island 

133 foot 

134 fire 

135 laugh 

136 sour 

137 soft 

138 hair 

139 give 

140 travel 

141 tobacco 

142 date 

143 cut 

144 long 

145 like 

146 playing 

147 people 

148 fazy 

149 important 

150 earth 

151 doctor 


Page 

No. word 

152 white 

153 water 

154 sillv 

155 sad 

156 cold 

157 child 

158 hit 

159 hate 

160 vacation 

161 try 

162 butter 

163 bread 

164 slow 

165 sky 

166 castle 

167 camp 

168 mischief 

169 love 

170 sweet 

171 street 

172 cotton 

173 correct 

174 man 

175 make 

176 stem 

177 spider 

178 protect 

179 power 

180 coat 

181 choose 

182 must 

183 mouth 

184 red 

185 quiet 

186 priest 

187 pray 

188 rough 

189 ring 

190 need 

191 nature 

192 science 

193 salt 

194 nut 

195 nice 

196 Sunday 

197 stomach 

198 dream 

199 beat 

200 dog 

201 dislike 
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Section III 

The Development of the Scheme for Scoring the 
Association Test 

Though it might be within the power of some peculiarly perspi- 
cacious individual to look at each response word and to surmise 
the nature of the probable associative process so accurately as to 
enable him to assign scores for (1) Courtesy, (2) Fair Play in 
Games, (3) Honesty in School Work, (4) Loyalty to his Fellows, 
(5) Mastery, (6) Poise, (7) Regard for Property Rights, and (8) 
School Drive (each as defined in Section II), the writers credited 
themselves with no such ability and have resorted to a strictly 
objective method to determine the credits that should be attached 
to the different responses. 

As has already been explained the judgments of teachers and 
fellow pupils were utilized to obtain criterion scores for each of the 
eight traits. These were combined for each trait and so expressed 
that each grade group studied had criterion scores of mean value 
zero and standard deviation equal to 1.00. Consider the first trait 
only and some particular response R to a particular stimulus S. 
If ten pupils gave this response R (out of the total of 723) we may 
list the criterion scores for these ten. If, for the first trait, they are 
as follows, .8, .6, —.4, 1.0, .7, .0, .3, .0, 1.5, .5, the sum equals 5.0 
and the average is .5. Thus, if a child responds with the word R 
to the stimulus S, the most likely estimate of his trait 1 criterion 
score is .5. We accordingly attach a weight (slightly modified as 
will be explained) of .5 to this response word. This constitutes the 
“score” for this response R to this stimulus S for this first trait. 
Similar scores are computed for all other responses. 

One way of investigating the merit of such a scoring scheme is 
to determine what would result if all response words were the re- 
sult of chance so far as the trait is concerned. Clearly the ten in- 
dividuals gitdng this response would not all have criterion scores 
of zero, for if that were the case the response R is definitely indica- 
tive of mean trait 1 standing. If the response is in truth non- 
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significant we will expect the ten scores to yield a distribution 
which is a random distribution from a parent population (of 723) 
whose mean is zero and standard deviation 1.0. Only in so far as 
the obtained distribution differs from this normal expectation 
should we believe that there is evidence of actual correlation be- 
tween response R and trait 1. Thus a modification of such a value 
as the .5 just given is called for. This may be a very radical modi- 
fication if the number in the class giving the response R is small. 

For example, if a large number of pupils give the response i, 
then the mean value of their criterion scores Zo.i calls for no 
modification, it being the proper value to assign to this response 
when some new pupil gives it. If but two or three give the re- 
sponse j, then zo.j is not as trustworthy a score when attached to 
the response of some new pupil as is zo.i. In other words, Zo.j 
should be discounted. We herewith give the determination of the 
appropriate discounting factors. 

Let responses be designated by a, b, c, . . . i, j, . . . 

Let number of i responses « ni 

Let mean zo, i.e. criterion, scores of these = zo.i and let the 
Hi such scores be designated by Zo.i, and where necessary to dis- 
tinguish them by Zo.u, Zo.ii> • • •> Zo.ini. 

Let the zo score for an added i response, i. e. one not made 
by any of the nj subjects used in determining the weight, be 
designated Zo.i and if necessary by Zo.n, Zo.i 2 > • • • 

Remembering that zo are standard measures, so that Szq — 0 

and j^2z®o = LOO, let us determine the variance of estimate 

when Zo.i is taken as evidence of Zo.i. Let Zoo.i = true mean Zo 
score for an infinite population giving response i, and let <r*o.i =* 
the variance of such scores from Zco.i- Thus if Zoo.j were taken as 
evidence of Zo.i the variance error = When zo.i, having a var- 

iance from Zqd i of ^ is taken in lieu of Zoo.i, the variance error is 

ni 


**■* 0.1 + 


<r*o.i 


ni 


or 


ni+1 

ni 
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The weight that should be given to zo.t is inversely as the variance, 
i. e. it equals 

nt 

(ni + 1) 


The labor involved in getting an estimate of each o-^o.i for each 
of the 5000-odd responses is prohibitive. We must therefore 


assume them equal and weight each zq.i as 
the accompanying table. 


Hi 

m -f T 


as shown in 


m 

1 

2 

3 

4 

5 

Multiplier of Za.i 

.5000 

.6667 

.7500 

.8000 

.8333 


6 

7 

8 

9 

10 

11 

12 

13 

.8571 

.8750 

.8889 

.9000 

.9091 

.9167 

.9231 

.9286 


14 

15 

16 

17 I 

18 

1. 

20 

21 

.9333 

.9375 , 

.9412 

.9444 

.9474 

.9500 

.9524 

.9545 


22 

23 ; 

25 

26 

28 

29 

31 

32 

34 

35 

37 

38 i 

40 

41 

43 

.9565 

.9600* 

.9643 

.9677 

.9706 

.9730 

.9750 

.9767 


44 

46 

47 

48 

52 

S3 

57 

58 

62 

63 

67 

1 68 

69 

70 

79 

.9783 

.9792 

.9804 

.9821 

.9836 

.9848 

.9856 

.9868 


80 

90 

100 

s 

89 

99 

on 


.9883 

.9895 

1.0000 
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SCHEME FOR SCORING THE ASSOCIATION TEST 


We thus see that if the credit attached to a certain response is 
consequent to a criterion score of but one person out of the total 
of 723, this score is multiplied by .5, that is, discounted 50 per 
cent; if consequent to criterion scores of two persons, it is multi- 
plied by .67, that is, discounted 33 per cent; etc. 

Though the credits and debits computed by means of these 
weighting factors may be the best possible, it still should be 
observed that they will not constitute very reliable scores if the 
numbers giving certain responses are small. For an initial sample 
of 723, yielding 28 response words, — more accurately, res 
classes, — the average number giving a certain response is 26. If 
this adequately indicated the reliability with which a score is 
attached to a response, the final outcome in terms of the reliability 
of the scoring scheme would be excellent. It, however, does not, 
for this mean 26 is probably derived from many small classes and 
a very few large classes. The reliabilities of scores computed from 
these small classes are ordinarily to be represented by frequencies 
from 1 to 10, and the reliability of the two or three large classes 
is represented by 


and not by 


ni 

(ni -h 1) (7*0. i 


nt 

(Ui + 1) 


Since the zo.i means for these large classes are found by observar . 
tion to diflFer but slightly from .0, there is evidence that for these 
classes <7o.i is larger than for the remaining classes. If <ro.i = co, 
the scores based upon these large classes would have no diagnostic 
value,'' for all such scores would be .0. 

It thus appears that scores based upon large classes cannot be 
expected to have much diagnostic value, and that it is only as 
children reply with more or less unique response words that real 
character divergencies from the mean, if present, will be revealed. 
In short, the solution to this problem depends upon ability reliably 
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to attach credits and debits to small class responses. This can only 
be done if the initial sample is much larger than 723. 

An alternative procedure which has been suggested is to use 
judgments of psychologists in attaching credits and debits. The 
possibility of this is admitted, depending upon the insight of the 
psychologists. To date, this procedure has not been followed and 
it must always be considered secondary to the method outlined 
in the preceding paragraphs, wherein judgments have been utilized 
in grading children for character traits, not in grading association 
responses. The former is the normal channel wherein judgments 
operate, not the latter. 

Section IV 

Results of the Study to January 1, 1932 

Financial assistance from the Commission on Investigation of 
the Social Studies of the American Historical Association roughly 
sufficed to carry the study to January, 1932, and to the accom- 
plishment of the following things: (1) the completion of the first 
administration of the test; (2) the determination for the largest of 
the three groups studied of the scoring scheme for this grade level; 
(3) the determination of the reliability coefficients of each of the 8 
trait measures; and (4) the correlation of each with the correspond- 
ing 8 criterion scores. It became obvious at this point that the 
test was not sufficiently reliable to yield satisfactory individual 
measures. 

To detertninejts value as an instrument for group measurement 
the Carnegie Foundation for the Advancement of Teaching, 
throuj^^^jhe Graduate School of Education, Harvard University, 
provi#di^nds for the continuation of the study. By January, 
1932, the test had been given to 435 children of average school 
grade 5.0 (beginning of the fifth grade, — children were drawn from 
%.urth, fifth, and sixth grades), to 8(X) children of average school 
grade 7.0 (children drawn from sixth, seventh, eighth, and ninth 
grades), and to 629 children of average school grade 9.0 (children 
drawn from sixth, seventh, eighth, ninth, tenth, eleventh, and 
twelfth grades). 
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It will be noticed that there is a certain amount of overlapping 
in the classes from group to group. The ninth graders in the group 
of 800 children of average grade 7.0 had Stanford Achievement 
Test scores below 78, or lacking such scores were of chronological 
age 16 or above, whereas the ninth graders in the group of 629 
children of average grade 9.0 had Stanford Achievement Test 
scores of 79 or higher, or lacking such scores were 15 years of age 
or younger. The duller ninth graders were incorporated into the 
group of mean school grade 7.0 and the brighter into the group of 
mean school grade 9.0. Similar demarcations were made for the 
pupils of each other grade. Funds proved inadequate for working 
up the data for all groups, and data for the two smaller groups 
have been laid aside, though the association test responses are 
available, should it prove possible to study these at a later date. 

The original seventh-grade group was divided into two parts: 
723, which will be referred to as the standardization group, were 
used for the determination of the scoring scheme; and 67, referred 
to as the reserve group, were laid aside to be used after the scoring 
scheme derived from the 723 was available, to determine the relia- 
bility of each of the 8 measures and to determine the correlation of 
each with the respective criteria. The necessity for such a reserve 
group is obvious, for the scoring scheme for each trait was so built 
up as to make the correlation between the responses as scored and 
the criteria as given high. Clearly, having done this, we cannot 
then investigate the correlation with the criteria for these 723 
cases and consider the result fair as a statement in general. In 
fact it would be spuriously high, but how much too high would be 
difficult to determine, and is unknown. Because of this, it is be- 
lieved that such a correlation coefficient would have no value, and 
it has not even been calculated. Neither have any reliability 
coefficients been derived from this group of 723, as these would 
also be spuriously high. 

The reserve group of 67 is too small to give satisfactory measures 
of reliability, but it was made small in order to retain as many 
cases as possible in the essential undertaking of determining the 
scoring scheme. Even the standardization group of 723 is not of 
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sufficient size to serve well its purpose. Table I presents the relia- 
bility coefficients found for this group of 67, using the split-test 
method. The responses to the 198 stimulus words were divided 
into 12th8 as nearly as could be, the score on the odd 12ths con- 
stituting a one-half test score, and that on the even 12ths the other 
half test score. For each trait the correlation between these two 
was then stepped up by the Spearman-Brown formula, giving the 
reliability coefficients for the entire test of 198 words. 

TABLE I 

RELIABILITY COEFFICIENTS FOR RESERVE GROUP 


Courtesy 46 

School Drive 72 

Regard for Property Rights 74 

Poise 64 

Mastery 44 

Loyalty .56 

Honesty S2 

Fair Play 62 


It was upon the basis of these reliabilities that it was concluded 
that none of the measures was accurate enough to be used for 
individual measurement, but that probably some of them would 
be of service in group measurement. As reported later, reliabilities 
as high as these have not been found to maintain when delinquent 
boys are tested. Some of the reasons therefor have come to light, 
but a remedying of the test in the light thereof has not been pos- 
sible. The correlations with the criteria for this reserve group are 
presented in Table II. 


TABLE II 

CORRELATIONS BETWEEN ASSOCIATION TEST AND JUDGMENTS 
OF TRAITS FOR RESERVE GROUP 


Courtesy 22 

School Drive 26 

Regard for Property Rights 17 

Poise 23 

Mastery i 02 

Loyalty 10 

Honesty 28 

Fair Play .15 
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The low value for mastery might be accounted for on the basis 
of a halo effect in those whose judgments were incorporated into 
the criterion scores, for mastery is generally negatively correlated 
with the other traits. The generally low values of these validity 
coefficients are only in small part due to the unreliabilities of the 
criteria, and in view of the size of the reliability coefficients, they 
seem explicable only on the basis of chance. 

When calculating the reliability coefficients it was observed that 
the reliability of the first 16 words with the second 16 was higher 
than that of the next to the last 16 with the last 16. It was thought 
that this might be due to pupils’ getting lazy and developing a 
tendency to check some printed word, rather than to write in the 
first words that came to mind. To check up upon this point, a 
new group of 190 seventh-grade pupils in the Bryant Junior High 
School of Minneapolis, Minnesota, were given the test without 
booklets. The reliabilities found are as follows: 

TABLE III 

RELIABILITY COEFFICIENTS FOR 190 SEVENTH GRADERS 


Courtesy 44 

School Drive 57 

Regard for Property Rights. 48 

Poise 45 

Mastery CX) 

Loyalty 31 

Honesty 57 

Fair Play 56 


These, in fact, are a little lower rather than higher than the relia- 
bilities from the reserve group. Also the correlations with criteria 
for this group of 190 are low. 

TABLE IV 

CORRELATIONS BETWEEN ASSOCIATION TEST AND JUDGMENTS 
OF TRAITS FOR GROUP OF 190 SEVENTH GRADERS 


Courtesy 18 

School Drive 16 

Regard for Property Rights 12 

Poise 00 

Mastery 15 

Loyalty — ,10 

Honesty 12 

Fair Play 07 
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Criteria scores for these pupils were not based on as many judg- 
ments per pupil as for the reserve group, and accordingly they may 
be thought of as of lower reliability. Even so, the showing cannot 
be considered favorable. 

For the original sample of 723, the reserve group of 67, and the 
seventh grade group of 190, the subjects dealt with may be 
assumed, so far as character is concerned, to have constituted a 
normal population. It remained to be seen whether the test would 
differentiate greater differences in these character traits than were 
ordinarily to be found in normal school groups. One would expect 
that delinquent groups, Ethical Culture groups, and groups having 
been given special character training would be differentiated one 
from another, even by means of a test of reliability not exceeding 
.3. It seemed more important to determine this, using the available 
measuring instrument, than to attempt immediately to refine it by 
extending and improving the scoring scheme. 

The indicated lines for improvement were, (1) to modify the 
form of administration, eliminating all printed responses which by 
their presence might alter the pupiPs reaction; (2) to present words 
orally and by means of large-type flash cards, so as to reduce to a 
minimum any uncertainty or annoyance due to difficulty of com- 
prehension of the stimulus word; (3) to extend the scoring scheme 
to include less frequent words than those thus far provided for, but 

(4) this could be done satisfactorily only by greatly extending the 
number of subjects used in building up the scoring scheme; (5) to 
re-determine the traits to be measured, securing a number, prob- 
ably less than eight, which are more independent of each other 
than the present eight; (6) to secure criterion scores more uncon- 
taminated by halo effect. Changes (1) and (2) were made in giving 
the test to all delinquent and specially trained groups, and we 
believe with good results, for difficulties connected with adminis- 
tration were lessened. Though it was reasonable to expect that 
much improvement in the test would follow the doing of (3), (4), 

(5) , and (6), it nevertheless seemed best first to keep the instru- 
ment as developed and to try it out on more varied groups than 
those to which it was originally applied, with the assumption that 
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group measures only would be of sufficient reliability to lead to 
important conclusions. 

1 Section V 

Administration of Tests in 1932 

The use of test pads or books, having on each page fourteen 
printed responses to the corresponding stimulus word, did not 
shorten the time required to administer the association test. On 
the contrary, it appeared that looking through the printed list of 
responses actually added to the time required of those pupils who 
failed to find their own responses in the list. A few pupils became 
confused when asked to write the first letter of their response on 
one page and the entire word on the next page. Occasionally a 
child lost time by looking for his response on the wrong page. 
The high cost of printing, binding, and transporting the two- 
hundred page pads contributed, also, to the decision to abandon 
their use in favor of the simpler procedure of having each pupil 
write his response words in a four-page folder. 

Another difficulty which had come to light was that pupils 
sometimes failed to hear the stimulus word correctly. It seemed 
that it might be desirable to present the stimulus word to the eye 
and to the ear simultaneously, in order to avoid any possibility 
of a misunderstanding. Flash cards were therefore prepared, each 
about 14x33^ inches in size, with the stimulus words in standard 
lower case block letters that could easily be read at a distance of 
forty feet, the height of the humdrum letters, i, e. neither non- 
ascenders or descenders, being 15/16 inches. The administration 
of the test was then changed to have the examiner hold up before 
the class a flash card containing the stimulus word while he pro- 
nounced it twice distinctly. The number of the stimulus word 
appeared on the flash card and was spoken by the examiner, so 
there was little chance of the pupil failing to understand which 
blank on the test leaflet was to be used in recording his response 
word. 

As a device for directing the pupil’s attention away from 
stereotyped responses, the checking of response words was 
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(Page One of Test) 

SOCIAL SCIENCE 
WORD ASSOCIATION TEST 


Kum Grade or Obm . 

Afe at lait Urthday . . yean Teacher 

School . City 

DIRBCTIOSS 


What is the next word }ou think of alter each word is spoken? 
(V) io Iront o( it. K it is not in the list, write it on the dotted line. 

Woio A Worn B 


■ busy 

odd 

-circus 

— - among 

-city 

break 

- few 


• fire 

equal 

. (un 

— -ws— ■■ even 

• gang 

— family 

groitp 

.give 

hot 

great 


H it is in the list, put a check mark 


WohD C 
above 
actor 

astTTOomy 

comet 

dim 

dreams 

dust 

earth 


-jam 

-many 

■ men 

.noiae 

.Qumbar 

push 

tan 


-haU 

.it 

.money 

numbers 

part 

piecea 

river 


.guide 

.heaven 

.high 

meteor 

morning 

.north 

player 


■ school 


.theater 


e p i Wi Mi Is iwm I. im> 


split 

——spoilt 

. . tinaet 

—two 

a. swsw 


shitting 

sparHing 

nm 

twinUo 
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(Page Two of Test) 

DIRECTIONS 

What ii the next word you think of after each word ia ipoken? Write the word you think of on 
the dotted line after ita number. 

1 26 SI 

2 27 SI 

Etc. to No. 2 s. Etc. to No. 50. Etc. to No. 75. 


(Directions for Test) 

SOCIAL SCIENCE WORD ASSOCIATION TEST 
Instructions for Administering 

See that each pupil is comfortably seated at a desk or writing table and is 
provided with a pen or a pencil which is in good working condition. Tell the pupils 
that you are going to give them an unusual but very interesting "word exercise.” 
Provide each pupil with a copy of the test folder. Direct the pupils to fill in the 
blanks at the top of the page. Answer any questions that may be asked about 
what to write after "Grade or Class.” 

As soon as everyone has filled the blanks at the top of the page, read slowly 
and distinctly the following explanation: 

“This is an exercise in which you arc to tell the next word you think of after 
each word that I pronounce. I shall say a word and show you the same word 
printed on a card. You are to think at once of some other word. Any word will 
do, if it is the first word which comes to your mind after I pronounce my word, 

"For the first few times, you may look in the printed list for the word that has 
come to your mind. If your word is in the printed list, make a check mark before it. 
If your word is not in the printed list, write it on the dotted line at the bottom of 
the column of printed words. Be sure to check or write the first word you think of 
after I tell and show my word.” 

“Now we are ready to begin. I shall tell you and show you my word, and then 
as soon as some other word comes to your mind, you may write it or check it and 
hold up your hand. Under ‘Word A’ in the left-hand column, check or write the 
first word you think of when I say the word ‘CROWD.’ ” Be sure to pronounce 
the word distinctly and to show the right card. 

"Look down the column under 'Word A’ and check the word you thought of, 
•I you can find it. If it is not in the list of printed words, write it on the blank at 
the bottom of the list, and then hold up your hand to show that you are ready for 
another word.” 
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Pause until all but two or three hands are up. Give these pupils extra help 
and explanations, if necessary. "Do you ail understand what to do when I pro- 
nounce a word? First, you think of another word, and then you either check or 
write your word in the proper place and hold up your hand. Are there any 
questions?" Answer any questions as briefly as possible. 

“Under ‘Word B’ check or write the first word you think of when I say the 
word ‘DIVIDE.’ " Show the card with the word on it. “Hold up your hand as 
soon as you have finished." 

"Under “Word C' check or write the first word you think of when I say the 
word ‘STAR.’ ” Show the proper card. 

"Now turn the page. Perhaps you had better fold the page back so that you 
can see only the first three columns of lines, numbered from 1 to 75." 

"From now on you are to write the first word you think of after I give you my 
word. Be sure to write your word on the dotted line after its proper number. If 
you don’t know just how to spell your word, write it as you think it sounds. This is 
not a spelling test. Be sure to write the first thing that comes to your mind after 
you hear my word.” 

"After the number 1, in the left-hand column, write the first word you think of 
when I say the word ‘LAMP.’" Pronounce each word distinctly and show it on 
the flash card. Allow about ten seconds, if necessary, before going on with the next 
word. A careful watching of the slow writers will help determine the proper time. 
If a following word is pronounced when 90 per cent have finished writing the 
response to the preceding word, no undue hastening of the slow writers is involved. 

For the first eight or ten words, use the formula: "After the number 2, write 
the first word you think of when I say the word ‘RICH.’ " Be sure to give the 
right word and to show the right card for each number. 

After the routine has been established, the formula may be shortened to: 
"For number 10, the word is SCHOOL.” 

Two or more class periods may be required for completing the two hundred 
items in the test. Allow at least one rest period during the test. 


retained for the three illustrative stimulus words with which the 
test begins. The first page of the 8>^xl 1-inch test folder is repro- 
duced herewith, together with the top of the second page and the 
accompanying instructions to the examiner. 

The ratings by teachers were obtained in the manner described 
on page 349, except that each teacher followed a different order in 
her ratings of the eight traits. This variation in the order of trait 
ratings was made to break up any constant error due to having 
all pupils rated first for “Courtesy,” second for “Fair Play,” etc. 
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STIMULUS WORDS FOR SOCIAL SCIENCE WORD ASSOCIATION TEST 


No. 

Word 

No. 

Word 

No. 

Word 

No. 

Word 

1 

lamp 

51 

cottage 

101 

wait 

151 

white 

2 

rich 

52 

eagle 

102 

wild 

152 

water 

3 

ride 

53 

hurt 

103 

over 

153 

siliv 

4 

age 

54 

kitchen 

104 

note 

154 

saa 

5 

dress 

55 

pain 

105 

difficult 

155 

cold 

6 

stranger 

56 

platform 

106 

desire 

156 

child 

7 

sundown 

57 

light 

107 

sickness 

157 

hit 

8 

night 

58 

lion 

108 

shock 

158 

hate 

9 

ocean 

59 

crowd 

109 

black 

159 

vacation 

10 

school 

60 

dance 

110 

band 

160 

try 

11 

music 

61 

tire 

111 

seed 

161 

butter 

12 

needle 

62 

train 

112 

once 

162 

bread 

13 

restless 

63 

friendly 

113 

mountain 

163 

slow 

14 

river 

64 

habit 

114 

milk 

164 

sky 

15 

poor 

65 

smooth 

115 

evening 

165 

castle 

16 

prison 

66 

soldier 

116 

eating 

166 

camp 

17 

rain 

67 

king 

117 

thief 

167 

mischief 

18 

range 

68 

learn 

118 

swift 

168 

love 

19 

mother 

69 

flag 

119 

home 

169 

sweet 

20 

name 

70 

form 

120 

admire 

170 

street 

21 

church 

71 

join 

121 

companion 

171 

cotton 

22 

citizen 

72 

joy 

122 

cave 

172 

correct 

23 

pleasure 

73 

fair 

123 

pure 

173 

man 

24 

prefer 

74 

fault 

124 

sorrow 

174 

make 

25 

square 

75 

country 

125 

sing 

175 

stem 

26 

stage 

76 

deep 

126 

dark 

176 

spider 

27 

loud 

77 

snake 

127 

current 

177 

protect 

28 

marble 

78 

star 

128 

fairy 

178 

power 

29 

command 

79 

queer 

129 

exercise 

179 

coat 

.30 

cow 

80 

college 

130 

journey 

180 

choose 

31 

stove 

81 

control 

131 

island 

181 

must 

32 

summer 

82 

fond 

132 

foot 

182 

mouth 

33 

listen 

83 

honor 

133 

fire 

183 

red 

34 

luck 

84 

table 

134 

laugh 

184 

quiet 

35 

boy 

85 

easy 

135 

sour 

185 

priest 

36 

captain 

86 

enjoy 

136 

soft 

186 

pray 

37 

simple 

87 

master 

137 

hair 

187 

rough 

38 

sleep 

88 

moon 

138 

give 

188 

ring 

39 

blue 

89 

new 

139 

travel 

189 

need 

40 

butterfly 

90 

rose 

140 

tobacco 

190 

nature 

41 

trip 

91 

aim 

141 

date 

191 

science 

42 

useful 

92 

bat 

142 

cut 

192 

salt 

43 

handle 

93 

sheep 

143 

long 

193 

nut 

44 

hay 

94 

short 

144 

like 

194 

nice 

45 

cheese 

95 

delighted 

145 

playing 

195 

Sunday 

46 

city 

96 

diamond 

146 

people 

196 

stomach 

47 

rush 

97 

narrow 

147 

lazy 

197 

dream 

48 

silk 

98 

old 

148 

important 

198 

beat 

49 

watch 

99 

walk 

149 

earth 

199 

dog 

dislike 

50 

whistle 

100 

young 

150 
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Section VI 

Character Status of the Groups Tested 

If the various measures developed from the responses of pupils 
in North Carolina and in Washington, D. C., possess any practi- 
cal significance as indexes of social attitudes or character, it 
seemed reasonable to suppose that classes elsewhere would differ 
from each other in their mean scores in accordance with known 
differences in their social backgrounds and training. An attempt 
was made, therefore, to locate and test classes of about seventh- 
grade maturity in which the pupils were believed to possess 
highly desirable social attitudes and other classes of equivalent 
maturity in which there was reason to believe that the pupils 
possessed undesirable social attitudes. Unless the contrast in 
known attitudes among these classes were reflected by statisti- 
cally significant differences in their test scores, there could be 
little hope of attaining ultimate success in developing practical 
measures of social attitudes or character from responses to a 
free association test. 

One of the outstanding efforts at character development in the 
elementary schools is that of the Ethical Culture Schools of 
New York City. Through the kindness of the director. Dr. V. T. 
Thayer, three sixth-grade classes in this school were given the 
free association test. The special training which these classes had 
received has been briefly described for us by Miss Ethel C. Brat- 
ton, Principal of the Elementary Department: 

“In the Elementary Department, as in all of the departments 
of the Ethical Culture Schools, the pupils have a definite course 
of study in moral education. This course is given by means of 
stories, true and imaginary; by the study of the life and work 
of fine characters, past and present; and by the discussion of 
ethical problems which arise in the study of current events as 
well as m the daily lives of the children. In this way the pupils 
get a background of knowledge, understanding, and feeling, as 
well as an attitude of thoughtfulness toward moral questions 
and behavior. 
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"There are many opportunities given in the school for the 
children to practice right ethical principles as well as common 
courtesies, through their cooperative government, philanthropic 
work, and freedom in chdice and initiative in classroom work.’* 

Among the agencies which have been most active in stimulating 
attention to character education in the public schools is the 
National Child Welfare Association. Dr. Frank Astor, of this 
organization, suggested that Public School No. 93 in New York 
City would probably show as large effects of character education 
as any other school available. Through the kindness of various 
officials of the New York City Public Schools, permission was 
obtained to examine pupils in a number of schools, including 
No. 93. Four sixth-grade classes of this school were given the 
association test. The Knighthood of Youth program, which was 
being used in P. S. No. 93, may be described briefly as follows: 

Knighthood of Youth is a club program for pupils of the ele- 
mentary grades, emphasizing and building upon the natural 
interests which children have in stories of knights and ladies in 
medieval castles. Each school room using this program organizes 
itself into a club, which adopts its own colors, songs, motto, codes, 
and the like, and which attempts to build itself a record of socially 
desirable deeds or accomplishments. Each achievement is graphi- 
cally recorded on a wall chart as one more stone in the wall of 
their castle. Socially desirable behavior is held up to the pupil as 
an ideal of modern knights and as a duty to his club. 

Through the cooperation of Mr. K. H. Thompson of Broad- 
water, Nebraska, we were supplied with the results of the associ- 
ation tests for five sixth-grade classes of the public schools at 
Scottsbluff, Nebraska. We are informed that the State Depart- 
ment of Public Instruction of Nebraska has been stimulating the 
use of the Knighthood of Youth program through the publication 
of books and pamphlets since early in 1930, and that the schools 
at Scottsbluff have been participating in the program. It should 
be remembered, however, that the pupils in these classes had 
not been under the influence of this statewide program for more 
than a few years at the time the association tests were adminis- 
tered. 
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Professor R. W. Fairchild of Northwestern University sugges- 
ted to us that some unusually effective character training was 
being given in one of the junior high schools of Kenosha, Wis- 
consin. Through the kindness of Superintendent G. F. Loomis, 
we were able to examine 112 seventh-grade pupils in this school. 
Mr. Loomis describes the school as follows: 

‘‘This school, which has been in existence for seven years under 
the principalship of the same man, had devoted particular interest 
to the development of school ideals, pupil responsibility, and 
pupil growth through pupil participation. The principal and 
teachers have been particularly sympathetic and have felt that 
they were teaching children rather than subject matter. The set 
up of the school has provided for a measure of pupil responsibility 
in matters of direction and discipline, has devoted a portion of 
each day as an activity period to self-directed and self-controlled 
activities, clubs, etc. The children come quite largely from foreign 
and unfavored homes.” 

Superintendent Theodore Saam of Elgin, Illinois, assisted the 
authors in securing responses from 311 seventh-grade pupils in 
his schools. Regarding the special training these pupils had re- 
^^eived Superintendent Saam wrote as follows; 

“Prior to 1930 there was a very deliberate and definite pro- 
gram of character education in the Elgin schools which was 
carried on more or less successfully in all the elementary grades. 
The program pjovided that the first ten minutes of each day be 
devoted directly to character education. In the fall of 1930, 
however, the teachers were not interested enough to continue this 
type of instruction, although it was approved and permitted. 
Since 1930 there has been no specific program of character apart 
from the continuous training of character in tasks assigned and 

S uiding the life of a child during the time he is under the juris- 
iction of the teachers.” 

There appeared in the New York Times for Sunday, August 28, 
1932, an article regarding the work of the public schools of Nor- 
walk, Conn., under the headline “City Molds Character.” In 
the article Superintendent John Lund wrote as follows: 
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“Our classroom techniques may be briefly summarized as 
emphasizing pupil activity, with the attention of the teacher 
focused on individual development in terms of an integrated 
personality rather than ujion the acquisition of book knowledge 
and skills.” 

Mr. Lund very kindly allowed the authors to examine 462 
seventh-grade pupils in the Norwalk schools, but in doing so he 
called attention to two other statements in the Times article: 

“We have consciously tried to avoid anything that could be 
construed as a definite character education plan.” “We make 
the emphasis on character development a major preoccupation 
for all teachers in all parts of the educational scheme.” 

In order to have a sampling of pupils who were known to be 
deficient in desirable social attitudes, permission was obtained 
to examine pupils in certain parental and probationary schools 
in New York City. The boys in these schools had all been com- 
mitted to them by court order because of misdemeanors or 
delinquency of one sort or another. The sixth, seventh, and 
eighth grades of Public School No. Ill, a parental school in 
Flushing, of Public School No. 61, a probationary school in 
Brooklyn, and of Public Schools Nos. 37 and 174, probationary 
schools in Manhattan, were therefore examined. These parental 
and probationary schools were assumed to contain children of 
less than average social character, while Public School No, 93 
and the Ethical Culture School were assumed to contain children 
of more than average social character. At the time the tests 
were administered the authors were uncertain how they should 
classify the character status of pupils tested in schools outside 
of New York City. 

Section VII 

The Scoring of the Tests Used in the Survey Program 

For the free association test to become a generally serviceable 
instrument, it is necessary to reduce the labor cost of scoring it. 
Roughly, a sample of seven hundred seventh-grade pupils will 
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give from fifty to one hundred and fifty different responses to a 
single stimulus word. A larger sample would yield additional re- 
sponses. Each different response may be expected to hold a unique 
story as to the mean outlook and associative tendencies of those 
so responding. If to the stimulus “pirate” one child responds 
“knife” and another “past,” it may be assumed that their interests 
and attitudes differ, or rather that the mean tendencies of the 
members of an entire group, all of whom respond “knife,” differ 
from those of a group all of whom respond “past.” Back of each 
response word is a correlative associative process characterizing 
the individual making the response. If we can but discover this 
mediating view, and credit or debit the response word in harmony 
therewith, we have to this extent a means of scoring a new subject 
who upon being given the stimulus word makes the same response. 
To grade two hundred response words for each of two hundred 
stimulus words upon each of eight traits calls for the determination 
of some (8X200X200) = 320,000 debits and credits, each one a 
statistical valuation based upon the trait ratings of the subjects 
giving the response in question in comparison with subjects in 
general. This is not only an enormous task to accomplish, but, 
once done, it yields so cumbersome a scoring scheme as to militate 
against its use. To abridge the process, a grouping of responses 
(later found to be too radical) was resorted to: 

A. The most frequent responses to a single stimulus, but 
never more than forty-four in number, were evaluated 
separately, yielding eight different scores for the eight 
traits studied, and 

B. Other responses, infrequent, but in the judgment of the 
chief clerk in charge judged to be very similar, were 
grouped together. For example, if to the stimulus “pirate” 
ten out of the seven hundred responded “weapon” and 
one or two responded “knife,” these would have been 
classified together. 

C. Failures to respond were treated as constituting a single 
class. 

D. All other responses were treated as a single class, desig- 
nated “unique.” 
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This last procedure (D) was dictated by considerations of econ- 
omy and time, and proved unfortunate. It was, of course, even 
at the start thought that this procedure would tend to lessen the 
reliability of the instruhient, but it was only after most of the 
statistical work had been done that it was discovered that it led 
to certain systematic errors as well. 

The following is offered as a surmise as to the nature of this 
error: When the members of the standardization group, composed 
of typical seventh-grade school children, gave a unique response, 
it was ordinarily because of a high type of association, e. g. “pirate” 
and response “racketeer,” or again, “pirate” and response “law- 
abiding,” with the result that the mean trait ratings were high, 
thus leading to the final result that a unique response was gen- 
erally given a positive credit for a majority, or in some cases for 
all of the eight traits. Thus, on the whole, unique responses were 
a good sign in the case of the standardization group. However, 
when delinquent boys in probationary and parental schools gave 
unique responses, they were probably consequent to a different 
type of mental process. Many of these responses were not only 
unique in the sense that other children (of the seven hundred 
studied) did not give them, but in the sense that they were utterly 
unrelated as judged by an intelligent adult, e. g. “pirate” and the 
response “street,” or “pirate” and response “go.” (One delinquent 
boy for a stretch of some fifty words responded in order with “I,” 


“don’t,” “know, 


,, «<T Jf (( 


don’t,” “know,” “I,” “don’t,” “know,’ 


etc.) 


To give a unique response made by a delinquent boy a generally 
high trait credit does not seem to be sound, though to give a 
unique response such a credit when the subject is a typical school 
child may be sound. At the time this hiatus in the significance of 
unique responses was discovered, it was far too late to attempt to 
remedy it, except as a partial remedy might be accomplished by 
rescoring all papers omitting all unique responses. (Even this lat- 
ter task proved to be an undertaking of several months’ time.) 
Obviously the real remedy is to be found in the elimination or the 
lessening of the unique-response class by (1) determining scores 
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for many words now classified as unique, and (2) grouping the 
rest in a number of carefully defined classes rather than bunching 
them together as of a single type. One such sub-class may well be 
composed of rhyming and other responses based upon the sound 
of the stimulus word. For example, “pirate” and the response 
“pie,” or “pirate” and the response “irate.” 

The first of the attached lists, in Table V, of unique responses 
contains samples drawn from pupils of the Ethical Culture 
School in New York City, who may be considered above average 
in such character traits as are here studied. Five response words 
were taken from the responses of each of twenty-live pupils taken 
at random. The second list, gotten in a similar manner, contains 
responses from pupils of probationary school No. 37 of New York 
City. An examination of single stimulus-response connections will 
scarcely reveal significantly different group tendencies, but re- 
membering that the total trait score of a pupil is a summation of 
198 credits or debits, it will be seen that a very small systematic 
difference will sum up to a substantial total. Admitting that the 
degrees of excellence of most of the individual associations are 
indistinguishable from group to group, still the writers judge that 
there is a preponderance of high type associations in the Ethical 
Culture group and of low type associations in the probationary 
school group. A study of this matter may be aided by an examina- 
tion of the differences in unique responses from the two groups to 
the same stimulus words. Table VI is a random sample of such 
responses. 

TABLE V 

UNIQUE RESPONSES, ETHICAL CULTURE SCHOOL 


Stimulus 

Response 

Stimulus 

Response 


age 

how 1 

smooth 

silverware 


music 

rhythm 

star 

shawl 


name 

everyone 

college 

team 


marble 

cdumn ' 

master 

sir 


iuck 

skill 

new 

address 


blue 

paste 

young 

singing 


vidnstle/ 

pipe 

kitchenette 

mountain 

thief 

sand 

liar 
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TABLE V (Cont.) 

UNIQUE RESPONSES, ETHICAL CULTURE SCHOOL 


Stimulus 

Response 

Stimulus 

Response 

citizen 

home 

diamond 

glisten 

command 

hard 

wait 

leave 

summer 

joy 

wild 

meek 

luck 

inherit 

sickness 

astonish 

simple 

nice 

eating 

dineing 

butterfly 

dainty 

home 

comfortable 

kitchen 

smoke 

note 

keynote 

light 

gleam 

admire 

ability 

lion 

viscious 

sing 

softly 

dance 

group 

exercise 

scales 

train 

learn 

foot 

weary 

habit 

rotien 

long 

teditious 

form 

games 

lazy 

daisy 

new 

brand 

earth 

God 

aim 

standard 

water 

fall 

young 

pep 

cold 

quits 

wild 

fierce 

hate 

bitter 

sickness 

not well 

bread 

worms 

pure 

real 

love 

pansy 

hair 

thread 

man 

many 

important 

needed 

make 

finnist 

child 

human 

stem 

tide 

hit 

collide 

must 

mess 

slow 

speed 

ring 

rasey 

coat 

aparall 

nut 

cake 

choose 

team 

Sunday 

7 day 

must 

you 

dog 

leash 

red 

corpasciles 

stranger 

fromafar 

quiet 

evening 

poor 

homeless 

priest 

cardinals 

rain 

shadows 

rough 

noise 

name 

everyone has 

nature 

dispossession 

citizen 

some cannot be 

poor 

less fortunate 

marble 

color 

salt 

spice 

luck 

much 

citizen 

belongs to city 

useful 

hand 

stove 

furnace 

hay 

slide 

dance 

performance 

city 

old 

smooth 

glide 

crowd 

heat 

form 

method 

form 

a 

fair 

rain 

join 

Red Cross 

country 

far ^ 

country 

Scarsdale 

deep 

earth 

honor 

your 

college 

coed 

master 

overseer 

table 

floor 

wild 

tangled 

easy 

very 

band 

strip 

enjoy 

myself 

pure 

cool 

moon 

change 

fairy 

light _ 
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TABLE V (Cont.) 

UNIQUE RESPONSES, ETHICAL CULTURE SCHOOL 


Stimulus 

Response 

Stimulus 

Response 

exercise 

foot 

give 

important 

skv' 

castle 

camp 

man 

mouth 

red 

need 

science 

stranger 

ocean 

music 

needle 

restless 

river 

poor 

rain 

ran He 

mother 

square 

loud 

lesson 

bottom 

part with 

needful 

horizon 

fortress 

Montana 

being 

cheek 

pink 

to 

astronomy 

shepheard 

cold 

flagllette 

blood 

uncomfortable 

sailing 

no nothing 

terreble 

what 

happy 

place 

ugly 

summer 

uck 

useful 

rush 

whistle 

platforms 

habit 

form 

country 

snake 

star 

control 

honor 

aim 

delighted 

note 

shock 

black 

hand 

seed 

once 

swift 

companion 

pretty 

nothing 

airplane 

subway 

grass 

convention 

want 

Fieldstar 

sweet 

"big 

lone 

wheel 

beat 

fine 

apisative 

that 

prove 

mist 

theater 

blossom 

life 

eagle 

brother 


UNIQUE RESPONSES, PUBLIC SCHOOL 37 

stranger 

hate 

island 

prison 

name 

right 

shock 

burn 

pleasure 

ease 

laugh 

straight 

boy 

boat 

travel 

tony 

city 

open 

important 

sereas 

luck 

prosper 

mischief 

done 

blue 

schady 

correct 

dome 

trip 

picnic 

power 

pull 

hurt 

bandage 

citizen 

countrymen 

friendly 

acquainted 

prefer 

go 

queer 

clear 

square 

even on four sides 

college 

trade 

summer 

fun 

control 

people 

simple 

nonotbing 

easy 

swim 

blue 

cloud 

enjoy 

him 

habit 

stop 

over 

jay 

insist 

queer 

nuts 

desire 

wild 

manica 

sickness 

reurt 

once 

Quarte 

fairy 


admire 



[ 381 ] 




TESTS AND MEASUREMENTS 
TABLE V (Cont.) 


UNIQUE RESPONSES, PUBLIC SCHOOL 37 

Stimulus 

Response 

Stimulus 

Response 


current 

nice 

doctor 

seuence 


exercise 

bones 

sad 

cannot 


laugh 

mouth 

cold 

very 


soft 

cat 

try 

what 


long 

pipe 

sky 

eve 


people 

eat 

sweet 

potato 


sing 

professor 

make 

cloth 


foot 

amutate 

protect 

armed 


cut 

illegible 

nice 

pal 


people 

bing 

dog 

fake 


lazy 

nogood 

music 

soft 


white 

plain 

poor 

dressed 


hate 

ate 

name 

somebody 


try 

rye 

blue 

sticky 


butter 

rye 

habit 

to keep on 


slow 

rotten 

smooth 

cleab 


mischief 

chief 

joy 

joy 


street 

earth 

form 

formation 


correct 

correction 

country 

fresh air 


stem 

unsterb 

star 

tree 


power 

unpower 

honor 

him 


mouth 

mother 

easy 

shoe 


rough 

unrough 

moon 

half moon 


beat 

fast 

rose 

agril 


ride 

move 

black 

polish 


music 

entertainment 

band 

misci 


restless 

movies 

pure 

reald 


name 

who 

sing 

micedar 


COW 

cud 

dark 

dierk 


boy 

- boy 

journey 

visitor 


train 

ecing 

island 

sourrand 


habit 

some 

foot 

far 


form 

do 

soft 

mice 


fault 

play 

people 

sombod 


country 

go 

cold 

frost 


fond 

save 

hate 

sea 


moon 

look 

Vacation 

a best 


wild 

bite 

cotton 

goings 


difficult 

do 

stem 

end 


shock 

Ash 

need 

bring 


sorrow 

surou 

Salt 

soft 


current 

courage ^ 

nut 

dope 


exercise 

tricks 

nice 

darling 


sour 

sour 

Stomach 

guts 


soft 

moss 

dream 

magic 


long 

story 

beat 

leg 


like 

others | 

dislike 

nogood 
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TABLE V (Cont.) 

UNIQUE RESPONSES, PUBLIC SCHOOL 37 


Stimulus 

Response * 

Stimulus 

Response 

stranger 

conral 

beat 

tree 

stage 

plarforel 

music 

radio 

luck 

close 

restless 

calm 

useful 

allwdes 

name 

unknown 

whistle 

signal 

prefer 

prefect 

form 

march 

stage 

baconly 

country 

live 

stove 

radiator 

honor 

complement 

boy 

son 

easy 

to do 

handle 

earn 

enjoy 

time 

hurt 

curt 

master 

dracula 

light 

bad 

diamond 

cool 

train 

subwey 

wait 

halt 

habit 

think 

wild 

lion 

join 

going 

once 

nremenber ■ 

fair 

chance 

fairy 

sissy 

fault 

SO 

exercise 

stretch 

star 

arice 

people 

citizen i 

queer 

to give in 

lazy 

him 

honor 

privilege 

sad 

vey losjsmen 

table 

place to lean on 

child 

daugther 

moon 

part of the earth 

sweet 

soft 

aim 

to get 

power 

over you 

sheep 

fleece 

need 

circle 

diamond 

coal 

beat 

rub 

seed 

orange 

night 

ni^ht 

mountain 

fell 

church 

private 

home 

lodging 

prefer 

among 

sing 

solo 

square 

equal 

give 

something 

light 

bad 

like 

long 

learn 

play 

lazy 

lazey 

country 

children 

slow 

sour 

honor 

to be 

street 

tar 

easy 

thing 

sorry 

science 

new 

just bought 

pray 

praise 

young 

future 

rough 

tumble 

shock 

noise 

dream 

feeling 

pure 

sterilized 

beat 

hot 

cut 

split 

sundown 

late noon 

water 

refreshing 

command 

charge 

hate 

unlike 

city 

down 

slow 

unfast 

rush 

clean 

sky 

work 

watch 

lookout 

stem 

edge 

friendly 

partner 

quiet 

quiteful 

smooth 

parm 

nature 

live 

form 

change 

dream 

dream 

fair 

even 
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TABLE V (Cont.) 

UNIQUE RESPONSES, PUBLIC SCHOOL 37 


Stimulus 

Response 

Stimulus 

Response 

snake 

nawwow 

cotton 

salt 

short 

not much 

protect 

body 

narrow 

skinny 

must 

due 

old 

long lived 

beat 

animal 

difficult 

unpr 

musical 

musical 

once 

hurry 

make 

make 

sour 

not good 

summer 

enjoyment 

tobacco 

war 

science 

still 

long 

give 

ride 

fruit 

like 

elect 

1 mother 

US 

lazy 

weak 

stove 

rock 

hit 

punch 

listen 

study 


TABLE VI 

RANDOM UNIQUE RESPONSES FROM ETHICAL CULTURE AND 

DELINQUENT GROUPS 

Stimulus 

Ethical Culture 

Probationary 

ride 

with 

move 

stranger 

relative 

hate 


shepaerd 

conral 


fromafar 


sundown 

bar 

late noon 

music 

rythym 

entertainment 

ti 

flagilette 

radio 

i4 


soft 

restless 

uneasy 

movies 

ii 

uncomfortable 

calm 

ti 

calm 


poor 

no nothing 

dressed 

U 

less fortunate 


ti 

homeless 


name 

everyone 

unknown 

a 

everyone has 

right 

u 


somebody 

church 

cathedral 

private 

citizen 

home 

countrymen 


French Revolution 


u 

belongs to city 


command 

hard 

charge 

stove 

furnace 

rock 

a 


radiator 

summer 

joy 

fun 

4* 

pretty 

enjoyment 
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TABLE VI (Cent ) 


Stimulus 

Ethical Culture 

Probationary 

luck 

nothing 

prosper 

<1 

skill 

close 

a 

much 


ti 

inherit 


simple 

nice 

nonthmg 

blue 

paste 

cloud 

t( 

leave 

schady 

trip 

picnic 

useful 

hand 

allwdes 

city 

busy 

open 


old 

down 

rush 

subway 

clean 

whistle 

mouthpiece 

pipe 

U 

signal 

H 

reed 


light 

gleam 

bad 

tram 

smash 

ecing 

t( 

learn 

subwey 

habit 

have It 

stop 


rotten 

think 


want 

some 

<( 


to keep on 

smooth 

silverware 

cleab 

it 

glide 

parm 


iceing 

do 

form 

golf 

U 

art 

formation 

it 

Fieldstar 

march 

a 

*( 

games 

change 

iC 

method 


join 

Red Cross 

going 

a 

help 


fair 

goodsport 

chance 


ram 

even 

country 

far 

go 

i( 

Scarsdale 

children 

iC 

sweet 

fresh air 

u 


live 

snake 

big 

nawwow 

star 

shalw 

tree 

i( 

lone 

arice 

queer 

quirios 

to give in 

ti 

clear 

it 


nuts 

college 

coed 

trade 


team 



women 


control 

wheel 

people 
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TABLE VI (Cent.) 


Stimulus 

Ethical Culture 

Probationary 

control 

levee 


« 

laughter 


honor 

my 

him 

H 

your 

complement 

<i 

beat 

privilege 

4t 


to be 

table 

floor 

place to lean on 

easy 

very 

to do 

44 


swim 

4< 


shoe 

enjoy 

myself 

him 

master 

overseer 

dracula 

44 

sir 


moon 

change 

part of the earth 

44 


look 

44 


half moon 

new 

present 

just bought 

44 

me 


it 

brand 


aim 

standard 

to get 

diamond 

glisten 

cool 

young 

singing 

future 

wait 

leave 

half 

wild 

fierce 

manica 

44 

meek 

bite 

44 

tangled 

lion 

sickness 

astonish 

rcurt 

44 

life time 


44 

not well 


shock 

burst 

burn 

44 

prove 

noise 

44 


fish 

black 

mist 

polish 

band 

theator 

miscl 

<( 

strip 


once 

an 

quarte 

44 

life 

hurry 

44 


nremenber 

mountain 

sand 

fell 

pure 

puries 

reald 

44 

teal 

sterilized 

44 

sweet 


current 

seed 

nice 

44 

^ea 

courage 

fairy 

glade 

straight 

44 

light 

sissy 

exercise 

reduce 

bones 

44 

scales 

tricks 

44 

lesson 

stretch 
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TABLE VI (Cont.) 


Stimulus 

Ethical Culture 

Probationary 

island 

sourrand 

prison 

t€ 

raft 


foot 

bottom 

amutate 

it 


far 

sour 

food 

sour 

long 

teditious 

give 

it 


pipe 

lazy 

unuseful 

no good 

tt 

daisy 

lazey 

ft 


him 

important 

needful 

sereas 

u 

laws 


n 

quick 


Cl 

needed 


water 

fall 

refreshing 

cold 

quite 

very 

Cl 

coat 


child 

human 

daughter 

« 

chide 


hate 

bitter 

unlike 

« 


ate 

<1 


sea 

slow 

speed 

rotten 

cc 


sour 

ti 


unfast 

sky 

sly 

eye 

u 

horizon 

work 

sweet 

strawberry 

potato 

street 

policeman 

earth 

make 

finnist 

make 

II 


cloth 

Stem 

tide 

edge 

it 


unsterb 

u 


end 

protect 

antagonzise 

armed 

must 

mess 

due 

it 

really 


it 

you 


mouth 

speak 

mother 

quiet 

tranquil 

quiteful 

rough 

splinter 

unrough 

« 

noise 

tumble 

K 

gentle 


need 

useless 

bring 

science 

Miss Klaer 

still 

nice 

disagreeable 

darling 

U 

mice 

pal 

dog 

leash 

fake 
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Though the processes of association are so subtle that it is not 
claimed the reader or the authors should be able to do so, never- 
theless it may be that one could judge by perusing these lists 
whether the character traits indicated by the one set of responses 
are generally higher than those indicated by the other. 

The authors believe that the preceding lists give support to the 
explanation offered, but this explanation is distinctly a surmise. 
It would seem that a grading of the unique responses of the Ethical 
Culture group or of typical schoo’ groups according to the best 
scoring scheme now available, wherein unique responses tend to 
receive positive credit, would be sound, and that omitting all 
grading of unique responses for the probationary groups would not 
be unfair to them. This dual procedure would lead to much more 
substantial group differences than those reported in Table X. 
The writers have not followed such a dual standard of grading, but 
they believe that the differences shown in Table X are decided 
understatements of the differences which would have been revealed 
had a more accurate means of grading the unique responses been 
at hand. 

Another, though less important, shortcoming in the test as 
given and as credited lies in the credits and debits given for 
“omitted” responses. No clearly systematic difference between 
delinquent and normal groups upon this basis has been discovered, 
except that more such omissions were found in the case of the de- 
linquent groups. This undoubtedly makes for lessened reliability, 
though perhaps it does not lead to a systematic error. This defect 
can be remedied by greater care in test administration. A teacher 
can observe that a child is writing nothing, but he is unlikely to 
discover while the test is in process that he is writing “I,” “don’t,” 
“know,” etc. 

Improvements both In the administration of the test and in its 
scoring, which will make for greater reliability and validity, seem 
to be definitely and specifically indicated as a result of the work 
to date. 
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THE SCORING OF THE TESTS 

The Scoring Technioue Followed (Hollerith) 

The task of assigning credits to different responses to each of 
the 198 stimulus words constituting the final test was a long under- 
taking. It finally resulted in tables, one for each stimulus word, 
giving for each of the more frequent responses eight different 
scores, one for each trait. Certain less frequent responses, judged 
to be very similar in the idea involved to certain of the more fre- 
quent responses, were given the same credits as those attached to 
the more frequent words. In the form in which they were first 
used in the actual scoring of pupils, these tables were typed on 
large cardboards, fifty to a board, so that the scorer was called 
upon to handle four of them in the scoring of a single test. For 
each response word the scorer would first find the response and 
then write down manually in appropriate columns the eight dif- 
ferent trait scores, except that zero scores were omitted. These 
columns were then added, yielding eight trait scores for the pupil, 
riie time to accomplish this with expert scorers was about one 
hour per test of 198 words. 

When the subsequent testing under the auspices of the Carnegie 
Foundation for the Advancement of Teaching was undertaken, a 
serious effort was made to reduce the cost of scoring, and in this 
the authors had the generous cooperation of the International 
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Business Machines Corporation, who adapted one of their tabu- 
lators to the problem in hand and built a special reading unit. 
The eight scores attached to a single response were punched upon 
one half of the card shown on previous page. 

The other half of the card was used for the next word in the 
alphabetical list of responses to the same stimulus word. The 
scores were multiplied by an appropriate factor to reduce their 
variability, enabling representation upon a 9-unit scale: 4, 3, 2, 
1, 0, — I, — 2, — 3, — 4. For each of the eight traits these reduced 
scores were punched upon the response score card, one column per 
trait being required. 

For each stimulus word there were not over twenty-two scoring 
cards, yielding not over forty-four different sets of gradings, as 
each end of the card gives a separate grading. For each stimulus 
word, these response score cards were filed in a separate holder, 
so that the ends bearing the response words and arranged alpha- 
betically were staggered and at all times visible. 

In scoring the response of a pupil, the scorer would note the 
pupil’s written response; draw the corresponding response card; 
enter it into the automatic reading device, one end if for the first 
word and the other end if for the next word as typed on the top 
of each card; press a lever, which would result in eight addends 
being printed and entered into the appropriate tabulators; remove 
the card from the reading device, and reinsert it in its original 
position in the file of response words. This was then repeated for 
the second word, etc., through the list of 198. Pressing a total 
button then gave the printed sums of the eight columns. 

In the case of expert machine operators, some 500 responses, or 
papers, were scored per hour. Though the time required to 
score a test is but two-fifths the former time, it is still a consider- 
able undertaking. With the increase in the number of response 
words scored, which is an important improvement definitely 
called for, the time cost of scoring a paper would probably be about 
thirty minutes rather ^han, as now, about twenty-four minutes. 
With this type of test, the scoring cost is, and is likely to remain, 
the major expense, but such expense as has been indicated should 
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scarcely be considered prohibitive if fairly trustworthy measures 
of character result therefrom. 


Section VIII 
Group Differences 

A number of points connected with the scoring of the free asso* 
elation test and with its reliability should be made clear before a 
discussion is entered into of differences in performance of classes 
from probationary and parental schools on the one hand, and of 
classes from schools having had some sort of character training 
on the other. The accompanying Table VII gives the mean test 
scores for 3 sixth grade sections of the Ethical Culture School of 
New York, and for 7 sixth to eighth grade sections from parental 
and probationary schools in New York. 

TABLE VTI 
MEAN S(X>RES 


Srhtiol 

Class No. of Cour- 
cases tesy 

Fair 

Play 

Hon- 

esty 

Loy- 

alty 

Mas- 

tery 

Poise Prop. School 
Rts. Drive 


6' 

20 

14.8 

15.7 

21,6 

15.2 

4.5 

14.4 

19.8 

18.6 

Ethical 

6* 

24 

17.8 

20.0 

24.6 

19.8 

4,8 

19.0 

23.1 

21.9 

Culture 

6* 

21 

15.5 

21.0 

24.2 

14.6 

5.6 

20,0 

19.3 

17.0 


6 

SO 

3.8 

8.7 

8.0 

9.3 

4.8 

8.3 

8.3 

.7 

l‘. S. Ill 

7 

60 

3.6 

8.9 

9.5 

8.7 

8.7 

7.3 

7.0 

4.9 

Parental 

S 

46 

3.7 

10.1 

II. 6 

8.3 

5.2 

5.0 

7.4 

8.2 


6 

19 

.7 

-1.8 

.3 

1.0 

4.8 

-.2 

-.9 

-3.4 

P. S. 174 

7 

12 

3.3 

2.5 

2.1 

.3 

4.1 

-2.3 

3.2 

1.8 

Probationary 

8 

17 

2.2 

2.8 

3,8 

7.6 

8.2 

.9 

5 4 

2.0 

P. S. 37 











Probationary 

6 

48 

-3.1 

.2 

-5.8 

3.0 

5.7 

-5.3 

-1.8-12.3 


The standard deviations of the distributions for the respective 
classes have not been calculated, but are approximately as fol- 
lows, which are the average values for the classes of Table X; 
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TABLE VIII 

APPROXIMATE STANDARD DEVIATIONS OF CLASSES 


Courtety 

Fair 

Play 

Honesty 

Loyalty 

Mastery 

Poise 

Prop. 

Rts. 

School 

Drive 

10.21 

10.24 

11 64 

8.92 

8.08 

11.52 

10.82 

11.05 


Table VII may be summarized into the table herewith. 


TABLE IX 

MEANS AND APPROXIMATE STANDARD DEVIATIONS OF 
MEANS FOR ETHICAL CULTURE AND PROBATIONARY GROUPS 



Size of 
Sample 

Cour- 

tesy 

Fair 

Play 

Hon- 

esty 

Loy- 

alty 

Mas- 

tery 

Poise 

Prop. School 
Rts. Drive 

Ethical Culture 

65 

16.1 

19,0 

23.5 

16 7 

5 0 

17.9 

20.9 

19 3 

Estimated Stand- 
ard Error of Means 


1.3 

1.3 

1.4 

1.1 

1,0 

1.4 

1,3 

1.4 

Probationary 

groups 

252 

2.1 

6.7 

5.2 

6.6 

6.2 

3 2 

4.8 

.4 

Estimated Stand- 
ard Error of Means 


.6 

.6 

.7 

.6 

.5 

.7 

.7 

.7 


It would seem that the superiority of the Ethical Culture 
group for every trait except that of mastery was indubitably 
established. It, however, must be pointed out that there is a 
vitiating factor revealed by these means which had not been 
anticipated, so that differences between the groups are not as 
well established as this table would indicate. Otherwise expressed, 
the differences between the groups are not to be attributed alto- 
gether to differences in the traits as labelled, but in part to 
differences in the effective administration of the test. It is to be 
noted that the mean scores for the delinquent groups are positive. 
This is contrary to expectation because the mean score in the 
case of the normal children involved in the construction of the 
scoring scheme is zero, ^nd it is hardly to be expected that these 
delinquent boys are superior in these eight character traits to the 
children of the standardization group. After considerable investi- 

r3921 







GROUP DIFFERENCES 


fration, the reason for this was established, as mentioned in 
Section VII, as due to the positive credit received for unique 
responses, and to a very small degree to omitted responses. 
Accordingly all papers were rescored, omitting all pupils having 
40 or more omitted responses, and omitting the scores attached 
ro all unique responses. It may be believed that the more nearly 
normal, that is “like the subjects of the standardization group,” 
the more unfair to a child is this procedure. That is, the pro- 
cedure of omitting unique responses is probably more unfair to 
the Ethical Culture group than it is to the delinquent group. 
Accordingly, as noted in connection with Table VI, differences 
in the accompanying Table X between the groups are not only 
understatements of trait differences, but understatements even 
of the measures of them that would be obtained were the scoring 
scheme extended to yield differential scores for responses now 
classed as unique. 

While examining the mean trait scores of Table X, the reader 
should remember that unique responses have not been scored, with 
the result that a slightly negative trend is to be expected. The 
trend would also be negative in the standardization group if scored 
in this manner. 

Let us study this table both by rows and by columns. The chil- 
dren of the Ethical Culture School generally stand high in all traits 
except Mastery. The girls of P. S. 93 are high in all traits except 
Mastery, but expecially high in Courtesy, Honesty, and School 
Drive. This may be consequent to the Knighthood of Youth char- 
acter training which has been in thorough operation at that school, 
or it may represent a sex difference, for this is the only school 
measured which was composed entirely of girls. The children of 
Elgin are quite normal in most traits, but they have slightly low 
irores in Fair Play and Poise. Those of Kenosha are somewhat 
high in all traits, including Mastery. Those of Norwalk would 
seem to be about normal, as judged by the standardization group, 
though perhaps a little high in Loyalty and School Drive, and low 
>n Poise. Those of Scottsbluff would seem to be normal as judged 
by the standardization group. Those in the probationary schools 
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TESTS AND MEASUREMENTS 


test slightly low in all traits except Mastery, in which they test 
high, and Loyalty, where they test as high or higher than the 
standardization group. It may aga n be mentioned that probably 
the scoring scheme employed has been unduly favorable to this 
probationary group, but not to the other groups. 

Examining the data by traits we note that wide differences are 
found between schools in Courtesy, Honesty and School Drive, 
but relatively small differences in Mastery. In so far as the test 
can be tested, it is a rather interesting commentary that ordinary 
school children, even those who have been subjected to a special 
character education program, do not attain as high a score in 
"Mastery” or “dominance” as do delinquent boys. This may not 
be an entirely fair appraisal, because the trait may be sex linked, 
and the probationary groups were composed of boys only. There 
is also the possibility that "character training” produces more sub- 
missive attitudes and conduct, or that many of the boys in the 
probationary and parental classes had been sent there because of 
"masterful” conduct that would not be present in “young gentle- 
men.” 

The reason for the relatively high standing of three groups in 
Loyalty, and the mediocrity of the other groups in this trait, as 
measured, is hardly to be attributed to chance. Apparently some 
sort of successful tuition has had a measurable effect in the three 
schools, Ethical Culture, Public School 93, and Kenosha. The high 
correlation of this trait with School Drive may suggest a single 
nuclear trait characterized perhaps by conscientiousness and loy- 
alty to self-imposed obligations. Many other group differences than 
those mentioned are undoubtedly significant, in view of the size 
of the standard errors of the means, but their causes are quite 
unknown to the writers. 

As part of an undertaking to establish a social studies curri- 
culum, the tuitionaldifferences'undoubtedly present in the different 
schools studied, whichvhave led to the significant trait differences 
as measured, should be a prime object of investigation. Such a 
study lies beyond the purpose of the authors, which has been to 
provide a measuring instrument which would serve as a tool for 
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SUMMARY 


ihe revelation of character differences. The test apparently accom* 
plishes this purpose when groups are compared. It is clear that it 
does not do so;for individuals. It is hoped that use will be made 
of the present instrument, without improvement, in studying group 
differences in character traits, and finally that the instrument may 
be improved to the point where it may be used in the future in 
the measurement of individual differences. 

Three suggestions, the outgrowth of the later work with the test, 
may be added to the six of Section IV for improving the adminis- 
tration and scoring of the free association test so as to yield a more 
reliable and a more diagnostic instrument. From an analytical 
study of the eight traits, which is not as yet completed, it is ap- 
parent that they are far from being independent one from another. 
Because of this, (suggestion 7) great improvement could be brought 
about by a new selection of traits so that a small number, perhaps 
not over three or four genuinely independent characters, could be 
dealt with. (8) An increase of the number of stimulus words, fol- 
lowed by a selection of those found to be the more diagnostic, 
should yield a test with even fewer than 200 words, which would, 
nevertheless, be much more accurate in its measurement. (9) 
Having data upon the significance of the many responses to many 
words, it should be possible to build up a semi-controlled associa- 
tion test involving the more significant stimulus words and the 
more significant only of the responses thereto, which test would 
have less irrelevant material in it than does a strictly free associa- 
tion test. 

Section IX 
SUMMARY 

This investigation has been an attempt to develop practical 
techniques for measuring social attitudes or character. The associa- 
tion test was selected for experiment because it seemed unlikely to 
be amenable to falsification through conscious attempts of persons 
being examined to modify their scores in directions which mi(.ht 
for any reason be thought advantageous. Although the techniques 
of the investigation would be equally appropriate with more ma- 
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ture individuals, this study has concerned itself with children of 
approximately seventh-grade maturity. Eight social attitudes or 
character traits commonly named as desirable were chosen for in- 
vestigation. 

For the classes involved in the derivation of the scoring scheme, 
the members of each class who stood high and the members who 
stood low w'ere defined for each trait by means of ratings by teach- 
ers and anonymous reports by fellow pupils. For each trait, re- 
sponses typical of the high group were positively credited, and 
those typical of the low group negatively credited in building up 
the scoring scheme. These scoring keys, developed from the re- 
sponses of seventh-grade pupils in North Carolina and the 
District of Columbia, were then tested on a research group yielding 
coefficients of reliability from .46 on Courtesy to .74 on Regard for 
Property Rights, and correlations with teacher and pupil ratings 
from .02 on Mastery to .28 on Honesty, and upon a second group 
consisting of junior-high-school pupils in Minnesota, yielding co- 
efficients of reliability ranging from .00 in “Mastery” to .57 in 
“Honesty in School Work” and in “School Drive,” and correla- 
tions with pupil and teacher ratings ranging from — .10 in “Loy- 
alty” to .18 in “Courtesy.” These results were distinctly discourag- 
ing, although numerous points had come to light at which definite 
improvements in procedures and techniques could easily be made. 
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Parsimony in the Judgment and Measurement 
OF Character Traits 

Truman L. Kelley 


The material studied and reported in the chapter upon Measure- 
ment of Certain Traits of Character provided data for an analyti- 
cal study of character traits. We may be interested in the follow- 
ing issues: 

(a) To what extent are the eight traits as judged independent 
one of another, or if not independent, to what extent can inde- 
pendent traits, eight or less in number, be used to characterize the 
same children, and how are these new traits related to the eight 
upon which judgments were made? 

(b) We may ask questions of similar import dealing with the 
fight character measures resulting from scoring the association 
test. 

(c) Thirdly, we may investigate to see if there are similarities in 
the analyses of the judgment measures and of the association test 
measures. We should expect similarity if our techniques are sound, 
for the scoring procedure was built up so that the association test 
scores would reflect the judgment ratings in so far as possible in 
the case of the group employed in determining the scoring scheme. 
However, since there are halo effects in the judgment data (see 
p. 434 for still another effect) and correlation between errors 
in the scoring of the association test, — the same words, not differ- 
ent ones, being scored in eight different ways to get the trait meas- 
ures, — it is clear that there is much need to investigate the ade- 
quacy of the procedures involved which were designed to allow 
properly for these systematic errors in the two types of measures. 

(d) Finally, we may ask if there is evidence suggesting that a 
smaller number of new traits be employed, — ^whether judgments 
constitute the original data or association test scores. 

The data herein used are those for the reserve group of 67 
and the Elgin group of 311 because none of it was involved .in 
determining the scoring scheme. 
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As a preliminary approach let us note a few correlations cor- 
rected for attenuation for the group of 311. These are given in 
the upper left portion of Table XII. Property Rights and Courtesy 
have an intercorrelation of l.(X)7 (the excess above 1.00 may 
readily be attributed to errors in the measures involved), thereby 
indicating practical dentity of these two traits as judged. Let us 
note the correlation of each with each other variable. We have 
the following: 



Fair 

PJay 

Honesty 

Loyalty 

Mastery 

Poise 

School 

Drive 

Courtesy 

.958 

.910 

.905 

— .429 

.808 

.886 

Regard for 
Prop. Rts. 

.940 

.924 

.919 

— .311 

.80S 

.932 


As a further check upon the possible value of the techniques 
employed, the mean scores of pupils in certain parental and pro- 
bat onary schools of New York City were compared with the mean 
scores of pupils in the Ethical Culture School and in other schools 
in which there was reason to believe the pupils possessed better 
than average social attitudes. The differences between the mean 
scores in the two extreme types of schools were several times the 
standard errors of these means, indicating that these differences 
were statistically significant, except in the trait called “Mastery,” 
in which the classes from the probat onary schools showed higher 
scores than did the classes from schools in which definite char- 
acter education had been given. Whether this trait called “mas- 
tery” or “dominance” is really a desirable social attitude is per- 
haps not finally determined, but this was the only one of the eight 
traits in which “bad” pupils equalled or surpassed “good” pupils. 

A study of the tables of mean scores of pupils of various classes 
in conjunction with the character education programs employed 
and the presumptive levels of the groups in the traits reveals evi- 
dence of real group differences, but even more strikingly such study 
shows that wide group differences in these character traits are the 
exception. Perhaps this is equivalent to saying that most char- 
acter education programs are rather ineffectual in modifying chil- 
dren. So broad a conclusion as this does, of course, call for further 
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careful investigation. The evidence supports the belief that some 
programs have greater effect than others, but we believe it hardly 
warranted at this time to attempt to list the character education 
programs upon the basis pf their effect upon each of the eight 
character traits. If any such attempt were made, we should be 
called upon to conclude that the life of the delinquent boy con- 
stitutes a good training in Mastery or dominance. We are some- 
what hesitant to do this, or to draw conclusions concerning char- 
acter education programs as related to the superior showing of 
certain “good” groups in Courtesy, Honesty, Poise, Regard for 
Property Rights, and School Drive. Though certain conclusions 
of this sort are probably warranted from the data, our knowledge 
of the need for improvement of the instrument, and our awareness 
that even such differences as are revealed are small in comparison 
with the variability of the groups studied, inclines us to urge 
further study before conclusions of broad import with reference to 
character education programs are drawn. 

The authors are disappointed, of course, that the weights they 
assigned to the responses to the words in their association test do 
not yield measures of social attitudes of sufficient reliability and 
validity to justify their use in measuring the social characters of 
individual pupils. They believe, however, that both the reliability 
and validity of such measures can in future studies be greatly im- 
proved, and they recommend the use of the test as it stands for 
further experimental work in measuring gross differences between 
junior-high-school classes in the following traits: Courtesy, School 
Drive, Regard for Property Rights, Honesty in School Work, and 
Fair Play in Games and Contests. The association test, even as it 
was scored in this study, is capable of distinguishing among junior- 
high-school groups that differ in these traits, and it should be pos- 
sible in experimental situations to measure by means of it the 
relative amounts of change produced in the character traits of 
classes by different types and amounts of training, environment, 
and the like, provided these amounts of change are of the order 
of, say, one-half a class standard deviation. 

Both by the correlation of the two variables and the correlation 
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of each with each other variable, we are warranted in believing 
that instead of two measures we have for all practical purposes 
but one. This seems surprising in view of the dehnitions of the 
traits as provided for the judges. Courtesy is a trait whose ex- 
tremes are characterized in the following words: 

(Potitive) 

Coniiderdte, courteous, polite, gracious, diplomatic, refined, well' 
bred, tactful, gentlemanly (ladylike), chivalrous, urbane. 

(Negative) 

Inconsiderate, discourteous, impolite, rude, unmannerly, undiplo- 
matic, crude, ill-bred, tactless, ungenriemanly (unladylike), bar- 
barous, boorish, rustic. 


and Regard for Property Rights is a trait whose extremes arc 
characterized as follows: 

(Positive) 

Always recognizes an owner’s right to control and to protect his own 
property. 

Asks permission before he borrows anything; returns things promptly 
and in good condition. 

Avoids injury to property of others, but tries to make good any 
injury done accidentally. 

Tries hard to find the owner in order to return anything he has found. 

(Negative) 

Takes things he wants, regardless of who owns them. 

Forf^ets to return and does not take care of things he has borrowed. 

Deliberately defaces and injures the property of the school and 
of others. 

Runs away and denies responsibility when he injures someone's 
property. 

Keeps an}'^thing he finds, even when he knows to whom it belongs. 


No single judge of the many who made judgments volunteered 
the idea that the two traits were the same. Nevertheless they 
turn out to be. Where different terms are employed there seems 
to be established the conviction that different things must be 
involved, though at times, as here, we have evidence indicating 
this not to be the case in any important sense. Whatever differ- 
ence there is is trivial in comparison with the sameness that is 
present. If Courtesy and Regard for Property Rights are not 
psychologically two traits but one, we are immediately led to 
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ask if some of the other measures may not merely be dependent 
functions. It is this more general problem which is here attacked. 
The endeavor is to see the extent to which some smaller number 
of variables than eight are adequate to account for all the sorts 
of variability represented by the initial eight variables. 

I'he method of analysis used is a modification of that presented 
hy Dr. Harold Hotelling.’ The purpose to be accomplished and 
rhe steps for doing it ma}’ be summarized as follows: (a) Original 
data are given showing the correlation between m variables (in 
the first study here reported eight judgment character scores for 
the reserve group of 67). Considering each variable equally im- 
portant, we assign a variance of 1.00 to each. The situation then 
has a total variance of m. (b) Various combinations of these vari- 
ables, or if each variable represents a dimension in m-dimensional 
space, various rotations of axes, may be investigated. No more 
and no less, except in exceptional cases, than m new variables, 
or axes, are needed to exactly characterize or duplicate the initial 
situation in its sundry sorts of variability. Variability in the direc- 
tion of the old axes is as given in the second column below and 
along the new axes in some manner, say as given in the 4th 
column. 

TABI.E I 

ILLUSTRATION (HYPOTHETICAL) FOR FIVE VARIABLES 


Initial Five Variables 

Five Derived Variables 

Variables 

Variance* 

Variables Variance* 

1 

1.00 

a 

1.30 

2 

1.00 

b 

.50 

3 

1.00 

c 

2.15 

4 

1.00 

d 

1.00 

5 

1.00 

c 

.05 

Sum 

= 5 00 


Sum = 5.(X) 


*The ‘ ‘variance” is the squared standard deviation. 

Variable a is some weighted linear combination of variables 
1. 2, 3, 4, and 5, and variable b some linear differently weighted 

'Hotelling, Harold: ‘‘Analy.sis of a Complex of Stati-stical Variables into Prin- 
cipal Components.” Journal of Educational Psychology, Vol, XXIV, Nos. 6 and 7, 
September and October, 1933. 
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combination of the same initial variables, etc. No information 
is given by variables a, b, c, and e above that to be found in 
variables 1, 2, 3, 4, and 5, It is merely differently organized. 
Various conditions ■which may be imposed on the new variables 
and which it would seem 'will greatly Facilitate thinking are, first, 
that a, b, r, d, and e be independent of each other. The initial 
variables are of course correlated by some complex pattern 
measured by ten intercorrelation coefficients, but quite beyond 
the mind to picture. The table of intercorrelations and variances 
for the initial variables is as follows: 


TABLE II 

INTER CORRELATIONS AND VARIANCES 






Initial Variables 





1 

2 

3 

4 

5 


1 

1.00 





2 

2 

rji 

1.00 




•S 

> 

3 

rii 

fti 

1.00 



'S 

4 

ri4 

Tti 

ri4 

1.00 


"5 

5 

rii 

Tti 

Tift 


1.00 

Variances 

1.00 

1.00 

1.00 

1.00 

1.00 

The table for the derived variables is 






TABLE III 





‘ INTERCORRELATIONS AND VARIANCES 






Derived Variables 





a 

b 

c 

d 

e 

n 

a 

1.00 





jS 

*c 

b 

.00 

1.00 




"T3 

c 

.00 

.00 

1.00 



.S! 

C 

d 

.00 

' .00 

.00 

1.00 


& 

e 

.00 

.00 

.00 

.00 

1.00 

Variances 

1.30 

.50 

2.15 

1.00 

OS 
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Clearly, all of one's thinking processes become simplified if 
variables o, bt c, dy and e are used instead of variables 1, 2, 3 4, 
and 5. 

However, the imposition of the condition of independence 
between variables is not sufficient to uniquely determine them, for 
an infinite number of new systems, such as that of variables a, 
b, c, d, and r, can be derived from variables 1, 2, 3, 4, and 5, all 
showing independence between measures. We may impose 
another condition. What this other condition should be may 
be consequent to psychological considerations or even physiological 
or social, but until such are made explicit and their value shown, 
a very real consideration from the standpoint of simplicity of 
thought is the principle of parsimony. Thus if our new variables 
are A, B, C, D, and E, such that A accounts for as great a pro- 
portion of the initial variance as possible, B for as great an amount 
of what is left as possible, etc., we have a system which still 
further simplifies thinking. The table is as follows: 


TABLE IV 

INTERCORRELATIONS AND VARIANCES 





Derived Variables 




A 

B 

C 

D 

E 

A 

1.00 





B 

.00 

1.00 




C 

.00 

.00 

1.00 



D 

.00 

.00 

.00 

1.00 


E 

.00 

.00 

.00 

.00 

1.00 

Variances 

3.00 

1.25 

.50 

.20 

.05 


Whereas the initial variables 1, 2, 3, 4, and 5, each included 
because it presumably measured some important aspect, were 
roughly equally important on the basis of the judgments that 
led to their inclusion and equally potent in terms of the complex 
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situation represented by the initial five variables, the derived 
variables A, B, C, D, and £ are not equally potent — variable A 
accounting for 60 times as much of the variance of the total 
situation as variable E. We may believe that variable E is not 
of the same importance as variable A in describing such individual 
differences as one is in general concerned with describing. This 
is, of course, an assumption, for no evidence is produced showing 
that variables A, B, C, D, and E do more richly characterize 
individual differences in situations going beyond those repre- 
sented by the initial variables 1, 2, 3, 4 and 5. 

The immediate aim of this study has been to use judgment 
measures of traits and to derive uncorrelated measures of the sort 
A, B, C, D, and E, wherein E can be neglected with trivial loss 
(in the hypothetical illustration, .01 of the total variance being 
thereby unaccounted for), D with a slightly greater loss (.04 of 
the total variance being thereby unaccounted for), etc. 

The method of principal components referred to has been 
followed with the following changes: instead of starting with a 
correlation table such as Table I, we start with the following 
table in which ri, r 2 , . . • are reliability coefficients. 

TABLE V 

INTERCORRELATIONS AND VARIANCES 

Variables 

1 2 3 4 S 


1 ri 

2 rt* Tt 

3 r,j r» Fg 

4 Tk r»« r *4 r, 

5 Tif Tit Tgt r4,. Ft 


Variances r, r» r, r, r* 


Total variance to be accounted for=Suni = (ri-i-r*-|-r*+r 4 -f-rt) <5.00 


In other words, we do not attempt to account for such portion 
of the variance as is due to chance, but only for such portion 
as is due to differences which are presumably real. 
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MEASUREMENT OF CHARACTER TRAITS 

When dealing with the intercorrelations between association 
test scores, we found that coefficients corrected for attenuation 
were systematically greater than 1. This was clearly due to 
correlation between error* in scoring, since the same pupil response 
items, not different ones, were scored to obtain each of the eight 
character measures. We removed this systematic tendency^ before 
writing down the initial table of intercorrelations to be analyzed. 
This added step has undoubtedly increased the size of the probable 
errors of our component variances and of the constants entering 
into the equations for obtaining the components, but we do not 
believe it has had any other effect. Unfortunately the probable 
errors mentioned are unknown. 

For the two groups the basic correlations are as given in Tables 
VI, VII, VIII, IX, X, XI, and XII accompanying. Variables 
1 to 8 inclusive refer to “traits as judged" by teachers and fellow 
students, and variables 11 to 18 inclusive refer to the same traits 
and in the same order “as scored” by means of the free association 
test. 

A comparison of the correlations of Table VI with those of 
Table VII will show many similarities and perhaps some discrep- 
ancies, but as a comparison of the correlated complex of traits 
as determined by the two methods of measurement will be much 
simpler when based upon the components found, and the weight- 
ings of the traits yielding these components, no immediate exam- 
ination is made of the correlation coefficients taken in pairs from 
these tables. 

An examination of the correlation coefficients of Tables VII 
and X reveals that many coefficients corrected for attenuation 
by the formula 

will exceed 1.00. Further, they will exceed 1.00 by amounts not 
attributabVe to chance. This is undoubtedly due to correlation 
between errors in the scores as derived. A correction for this has 

^See Appendix A at the end of this paper. 
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been worked out as given in Appendix A. Its application yielded 
Table VIII from Table VII, and the lower right quarter of Table 
XI from Table X. 


TABLE VI 

CORRELATIONS FOR RESERVE GROUP OF 67 






Traits at Judged 





1 

2 

3 

4 5 

6 

7 8 

Courtejy .... 

.. 1 

.891 






Fair Play. . . . 

.. 2 

.598 

.834 





Honesty 

.. 3 

.592 

.688 

.879 




Loyalty 

.. 4 

.486 

.565 

.644 

.879 



Mattery 

.. 5 

-.338 

-.105 

-.064 

-.107 .878 



Poise 

.. 6 

.460 

.571 

.617 

.644 -.036 

.837 


Prop. Rts 

.. 7 

.776 

.677 

.641 

.527 -.249 

.500 

.880 

Sch. Drive. . . 

8 

.647 

.611 

.777 

.574 -.017 

.545 

.711 .918 


TABLE VII 

CORRELATIONS FOR RESERVE GROUP OF 67 






Traits as 

Scored 





• 

11 

12 

13 

14 

IS 

16 

17 

18 

Courtesy . . 

. ...11 

.462 








Fair Play . . 

. ... 12 

.693 

.619 







Honesty . . . 

....13 

.620 

.698 

.525 






Loyalty. . . 

. ... 14 

.567 

.663 

.671 

.556 





Mastery. , . 

...15 

-.107 

.038 

.086 

.153 

.435 




Poise 

,...16 

.661 

.803 

.700 

.698 

.216 

.642 



Prop. Rts.. . 

...17 

.783 

.745 

.791 

.634 - 

-.138 

.645 

.739 


Sch. Drive. . 

... 18 

.778 

.584 

.724 

.569 - 

-.137 

.510 

.832 

.718 
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TABLE VIII 


CX)RRELATIONS, RESERVE GROUP OF 67, INTERCORRELATIONS 
MODIFIED TO ALLOW FOR CORRELATION 
BETWEEN ERRORS IN SCORING 






Traits as 

Scored 






11 

12 

13 

14 

15 

16 

17 

is' 

Courtesy 

11 

462 








Fair Play 

12 

429 

619 







Honesty 

13 

350 

467 

525 






I oyaltv 

14 

310 

474 

430 

556 





Mastcrv 

15 - 

- 002 

065 

107 

150 

435 




Poise 

16 

417 

602 

430 

481 

141 

642 



Prop Rts 

17 

526 

580 

630 

496 - 

- 102 

534 

739 


Sth Drive 

18 

542 

428 

520 

393 - 

' 136 

322 

686 

.718 


TABLE IX 

CORRELATIONS, GROUP OF 311 






Traits as 

Judged 






1 

2 

3 

4 

5 

6 

7 

8 ' 

(.ourtcsy 

1 

821 








Fair Play . 

2 

798 

846 







Honesty 

3 

739 

729 

802 






Loj alty 

4 

6% 

758 

689 

.720 





Mastery 

5 - 

- 319 - 

- 304 - 

- 186 

- 194 

673 




Poise 

6 

629 

642 

652 

637 

- 188 

738 



Prop Rts 

7 

835 

791 

757 

713 

- 234 

632 

.837 


Sch Drive . 

8 

758 

722 

782 

659 

- 183 

650 

.804 

.890 


TABLE X 

CORRELATIONS, GROUP OF 311 






Traits as Scored 






11 

12 

13 

14 

15 

16 

17 

18 ' 

Courtesy 

11 

280 








Fair Play 

12 

633 

.237 







Honesty 

13 

652 

.698 

380 






Loyalty 

14 

580 

664 

665 

286 





Mastery 

IS - 

- 307 

- 173 - 

- 125 

- 184 

.245 

425 



Poise 

16 

584 

726 

726 

678 

-.038 



Prop Rts 

17 

750 

.682 

.725 

604 

- 199 

.597 

.300 


Sch. Drive. 

18 

.704 

570 

674 

.567 

- 157 

.543 

.711 

.363 
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TABLE XII 

GROUP OF 311, SAME RELATIONSHIPS AS SHOWN IN TABLE XI EXCEPT THAT 
CORRELATIONS ARE CORRECTED FOR ATTENUATION 
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TESTS AND MEASUREMENTS 


Tables VI, VIII, IX, lower right hand quarter of XI, and XII 
have been analyzed to determine the major components in each. 
The reason Table XII instead of Table XI was studied is that the 
smallness of the reliability coefficients for the test scores as com- 
pared with the judgment ratings would of itself necessitate a much 
greater weighting in an analysis into components of the traits as 
judged in comparison with the traits as scored by the association 
test. Greater weightings would obscure the object of the analysis 
which is to ascertain similarities in the ratings in the traits as 
judged and the same traits as scored. Therefore in this instance a 
table of coefficients corrected for attenuation constituted the basic 
table. The issue of similarity between traits as judged and the 
same traits as scored is a theoretical one so that the correction for 
attenuation is warranted. But such correction has not been fol- 
lowed in connection with the other tables and should not be fol- 
lowed if the practical issue of how to combine measures and get 
component scores is being considered. It seems to the writer that 
the type of table which in general should be analyzed is Table V 
wherein initial variables are roughly judged to be of equal impor- 
tance. A very simple modification of the entries of such a table 
may be made if the initial variables are considered of unequal im- 
portance. If the judged weights are wi. w*, W| . . . all that is 
necessary is to multiply each reliability coefficient by the square 
of the corresponding weight and each correlation coefficient by the 
product of the two corresponding weights. 

Tables XIII, XIV, XV, XVI, and XVII herewith give the re- 
sults of the analyses of the various tables. Since in every instance 
the greater part of the variance is represented in two or three com- 
ponents, and since in the tables which have involved correction for 
correlation between errors and for attenuation, large chance errors 
in the analyses must be present, these have been carried to the 
determination of three or four components only, except in the case 
of Table VI where the analysis has been carried to the limit just 
to show what one complete analysis looks like and to emphasize 
the negligible importance of the last several components. The 
writer attaches no special significance, other than the fact that 
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they are small, to the weightings for components IV, V, VI, VII, 
and VIII. Certain imaginary variances are occasionally to be found 
in these tables, e. g., the residua l standard deviation for Poise, 
Table XIII, is given as V — .0001 The value should be .00. The 
imaginary value is due either to computational error or to an error 
in the obtained reliability coefficients which have led to a coefficient 
of correlation corrected for attenuation which is greater than 1.00. 

In the accompanying tables variables are numbered in the same 
order as in the preceding tables. 

The k-values give the variances of the various components and 
their sum equals the sum of the entries in the diagonal of the corre- 
lation table and is the total variance represented by the complex 
situation. 


TABLE XIV 

RESERVE GROUP OF 67, SCORES 


Components 


I 

II 

III 

Remaining 

components 

All 


k values 


3.521 

.719 

.281 

.175 

4.696 



Weights of variables yielding these 






components 


^Residual 







e’s 


Courtesy 

11 

.619 

-.115 

-.009 

.257 


Fair Play 

12 

.727 

.164 

-.213 

.132 


Honesty 

13 

.683 

.026 

.201 

.130 


Loyalty 

14 

.638 

.230 

.067 

.304 


Mastery 

15 

.041 

.580 

.294 

.100 


Poise 

16 

.692 

.312 

-.256 

.024 


Prop. Rts. 

17 

.849 

^-.195 

.004 

V-.0192 


Sch. Drive 18 

.732 

-.391 

.195 

V-.0091 



*St«i<l«tl deviatwii of the proportion of the initial variaUea unaccounted for by Conponenti 
I. II and III. 
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TABLE XV 

GROUP OF ill. JUDGMENTS 





Components 




1 

n 

III 

Remaining 

components 

AH 

k values 

5 211 

625 

212 

279 

6.327 


Weights of vatiablts yielding these 
components 

‘Residual 

t 


Courtesy 1 

Fair Play 2 
Honesty 3 

Loyalty 4 

Mastery 5 

Poise 6 

Prop Rts. 7 

Sell Drive 8 

887 
.887 
.857 
.811 
- 305 
760 
898 
877 

- 078 

- 067 

115 

058 

756 

077 

039 

137 

107 

- 048 

031 

- 159 

QOS 

- 333 

147 

198 

0 S 

.127 

.229 

231 

.184 

.090 

210 

090 

251 


‘Standard deviation of the proportion of the initial vanablca unaccounted for by Conpooenti 
I, II and III. 


TABLE XVI 

GROUP OF 311, SCORES 






Components 




I 

II 

111 

Remaining 

components 

All 

k values 

1.927 

411 

164 

.015 

2 517 


Weights of vanables yielding these 
components 

‘Residual 

it’s 


Courtesy 11 

.439 

-.236 

.042 

.173 


Fair Play 12 

.461 

.205 

- 031 

\/-.0188 


Honesty 13 

.630 

.005 

- on 

V- 0167 


Loyalty 14 

.521 

.082 

-.109 

V~ .0039 


Mastery IS 

-.086 

.389 

.295 

.010 


Poise 16 

.586 

.294 

- 119 

\/-.0190 


Prop. Rt*. 17 

.495 

-.109 

.148 

.146 


Sch. Drive 18 

.507 

- 240 

161 

.149 



‘Standard deviation of the proportion of the inital rariablei unaccounted for by Compooenu 

I. II ud 111. 
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TESTS AMD MBASUKEMBNTS 
TABLE XVII 


GROUP OF 311, JUDGMENTS AND SCORES 
COEFFICIENTS CORRECTED FOR ATTENUATION 


Component! 

I II III Remaining All 

component! components 

k value! 7.064 S.S2J 1.438 1.977 16.00 


Weight! of variables 

yielding theae components *Residual 


Courtesy 

1 

.822 

-.540 

-.015 

a s 

,183 

Fair Play 

2 

.801 

-.539 

.018 

.259 

Honesty 

3 

.857 

-.424 

.093 

.277 

Loyalty 

4 

.810 

-.493 

.131 

.288 

Mastery 

5 

-.184 

.446 

.276 

.831 

Poise 

6 

.821 

-.342 

.177 

.421 

Prop. Rts. 

7 

.80S 

-.565 

-.006 

.179 

Sch. Drive 

8 

.806 

-.454 

.090 

.369 

Courtesy 

11 

.640 

.540 

-.423 

.347 

Fair Play 

12 

.473 

.827 

.353 


Honesty 

13 

.646 

.804 

.009 

\/-.064 

Loyalty 

14 

.492 

.841 

.055 

.217 

Mastery 

IS 

-.209 

-.046 

.856 

.472 

Poise 

16 

.504 

.758 

.373 

.182 

Prop. Rts. 

17 

.601 

.660 

-.159 

.421 

Sch. Drive 18 

.632 

.577 

-.305 

.418 


'Standard deviation of the proportion of the initial variables unaccounted for by Componenti 
I, II and III. 

If the variances of the components are represented as per- 
centages of the total variance, we have, for Table XIII: 


Components I 

\II 

III 

IV 

V VI VII VIII 

Variances as percent- 
ages of total 66 

14 

8 

4 

3-f 3- \+ 1- 
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We see that a replacing of the initial variables by two new ones, 
component I and component II, will yield 80 per cent as much 
information about the variability of the pupils as do their ratings 
upon the entire eight traits. Of course appropriate names and 
descriptions for the new variables should be devised and ratings 
secured in harmony therewith before we can actually profit by 
using these components. This might prove a difficult under- 
taking but it surely seems worth attempting. Probably three 
components would suffice to tell all that is worth recording, for 
there is certainly much error in the weightings of the minor com- 
ponents, components IV, V, VI, VII, and Vlll probably having 
negligible real meaning, that is, they are probably not much other 
than chance. 

To get component I from the eight judgment scores (these 
having been reduced to standard measures), we employ the equation 
Xj= .800zi-l-.804zj-l'-856zi-}*.758z4 — .200zj-f'-728ze-|-.843z7 -l-.846zg 
X, are scores whose mean is zero and whose standard deviation 
equals \/^. Similarly we can obtain components II, III, etc., using 
the weights of the columns of Table XIII for these components. 
A study of the weights of component I leads us to consider that 
this component is a general average of all the traits except that 
the trait Mastery is omitted, and except that Poise and Loyalty 
are slightly underweighted in this average. If we now examine 
component I for the judgment scores for the group of 311, we 
see from Table XV that it may be characterized in the same 
words. The correlations between the weightings of the first com- 
ponent for the group of 67 and the first component for the group 
of 311 are shown graphically in Chart I wherein the numbers 
written near each point refer to the traits as numbered. Clearly 
nearly the same thing is represented by component I of Table 
XIII as is shown by component I of Table XV. Examining 
component I for the judgment scores and component I for che 
association test scores leads us to compare column I, Table XIII 
with column I, Table XIV, column I, Table XV, with column I, 
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Table XVI, and column I, Table XIV, with column I, Table XVl. 
Gathering these four sources of data together yields the following; 

WEIGHTINGS YIELDING THE FIRST COMPONENTS BY 
EACH OF THE FOUR ANALYSES 


Judgments Scores 

Variables • Variables 

N-67 N-311 N = 67 N = 311 

Courtesy 1 .80 . 89 . 62 . 44 11 

Fair Play 2 80 . 89 . 73 . 46 12 

Honesty 3 .86 . 86 . 68 . 63 13 

Loyalty 4 .76 .81 .64 .52 14 

Mastery S -.20 -.30 .04 -.09 IS 

Poise 6 .73 .76 .69 .59 16 

Prop. Rts. 7 84 .90 .85 SO 17 

Sch. Drive 8.. . .85 .88 .73 .51 18 


The relationships are shown graphically in Charts I, II, III, and 
IV. It is clear that component I from traits as judged and com- 
ponent I from traits as scored are very nearly the same thing. 
We need a name if merely for the purpose of the present discussion, 
for the thing nuclear to these four component I measures. Certain 
names which have been suggested for this trait are ^‘social minded- 
ness,” “social conformity,” “good citizenship,” “virtue.” Since it 
seems to be the epitome of all those virtues cherished by the 
American forefathers, and since we do not wish to be involved in 
the necessity of proving it identical with some trait already named, 
let us call it “Puritania.” 

Remembering that the chief consideration in the choosing of 
the eight traits by a consensus of competent people was that they 
were separately important, we see that the outstanding character- 
istic of Puritania is its sweep or generality. The certainty with 
which the first component can be isolated is a function of its 
magnitude and the ratio of its variance to that of the next smaller 
component. For all four 8-variable analyses the ratio of the 
vaj^]Kffl6f the first component to the second is greater than that 
of the Ifecond to the tl^rd, and thus the certainty with which we 
speak of Puritania will be greater than that with which we 
^an speak of any second or third component variable. However, 
these latter seem far from meaningless as will be apparent from a 
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Study of the weightings of the initial variables which yield them. 

The study of the weightings of component II, Tables XIII 
and XV shows in each instance a predominance of the judged 
trait Mastery. 

Tables XIII, XIV, XV, and XVI, from which the accompanying 
weightings have been drawn, and Charts V, VI, VII, and VIII are to 
be examined in order to make a study of the weightings of the second 
components as given by the judgment and the score variables. 

WEIGHTINGS YIELDING THE SECX)ND COMTONENTS 
BY EACH OF THE FOUR ANALYSES 


Judcments Scores 


Variables ■ — — — ■ . Variables 

N=67 N = 311 N = (J N-Hl 

Courtesy 1 —.32 —.08 —.12 —.24 11 

Fair Play 2 05 - .07 .16 .20 12 

Honesty 3 .17 .12 .03 .00 13 

Lovalry 4 IS .06 .23 .08 14 

Mastery 5 86 .76 .58 .39 15 

Poise 6 22 .08 .31 .29 16 

Prop. Rts. 7 -.19 .04 - 20 -.11 17 

Sch. Drive 8 IS .14 -.39 -.24 18 


Such a study indicates less similarity in the four second com- 
ponents than was the case in the four first components. It may be 
haiardovis to attempt to name a nuclear trait underlying these 
four components, though there is something heavily weighted with 
Mastery, slightly with Poise and Loyalty, and slightly and nega- 
tively with Courtesy and Regard for Property Rights, which is 
determinative of the nature or direction of this component. The 
slight element of lack of consideration and the slight elements of 
self-confidence and loyalty should be added to the concept of 
dominance well to characterize the nucleus of the four second 
components. They do, however, differ sufficiently one from another 
to leave considerable doubt as to the fitness of any name which 
might be chosen. “Rugged Individualism” is one suggestion 
offered to the writer and it will serve as a name for the present 
discussion, though upon study it may be found to be very similar 
Allport's Ascendency or Bemreuter’s Dominance. Its essential 
characteristics are, first, complete independence of Puritania, and, 
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second, an importance in accounting for the variability represented 
by the eight initial variables of about one in seven. The reader 
must firmly fix in mind that there is no antagonism between these 
first and second component variables. Possession of a large amount 
of the one presages nothing as to a probable amount of the other. 
Thinking in terms of these variables is peculiarly simple because 
while thinking of one the other can be entirely neglected. If with 
reference to a seventh grade child knowledge of the one is at hand 
and later knowledge of the other becomes available, it is com- 
pletely additive information calling for no discounting because in 
part already known. It is as though the child formerly one- 
dimensional now becomes two-dimensional. Such other dimen- 
sions of the child’s mental life as may be independent of these two 
(perhaps one or more types of ability) would be similarly con- 
tributive of entirely new information, but what these are and 
whether they exist or not cannot be foretold previous to an experi- 
mental study of them in connection with these two. 

A study of the third components resulting from the four analyses 
gives weightings of similarly named initial variables which are 
definitely and positively correlated, but the correlation is so low 
that doubt is cast upon any name which might be ascribed to all 
four third components. 

We will not give a name to this nucleus. The accompanying 
weightings suggest a certain instability factor, — lack of Poise, of 
Loyalty, and^f Fair Play, but accompanied by a certain vigor 
represented by the presence of Mastery and School Drive. 


WEIGHTINGS YIELDING THE THIRD COMPONENTS 
BY EACH OF THE FOUR ANALYSES 


Judgments 

Variables * * 




N-67 

N=311 

Courtesy 

1 

.22 

.11 

Fair Play 

2 

.03 

-.05 

Honesty 

3 

.01 

.03 

Loyalty 

4 

-.41 ^ 

-.16 

Mastery 

S 

.25 

.00 

Poise 

6 

-.36 

-.33 

Prop. Rts. 

7 

.24 

.15 

Sdu Drive 

8 

.25 

.20 


Scores 


N »67 

N = 3 n 

Vanables 

-.01 

.04 

11 

-.21 

-.03 

12 

.20 

-.01 

13 

.07 

-.11 

14 

.29 

.30 

15 

-.26 

-.12 

16 

.00 

.15 

17 

.20 

.16 

18 


[ 428 ] 




MEASUREMENT OF CHARACTER TRAITS 

Possibly one end of this trait is represented by schizophrenia and 
the other by a well integrated personality. This is merely a sug- 
gestion and in no sense a conclusion. 

The analyses thus far described have been of judgments on the 
one hand and of scores on the other. In the case of the group of 
311 we have all 16 measures for each pupil and have correlated 
every one with every other one, yielding a table of intercorrela- 
tions, Table XI, which could be analyzed to yield components 
running through all 16 measures. Such an analysis might be expec- 
ted to yield more conclusive components than the separate analyses 
of the judgment scores and of the association test scores. How- 
ever, an examination of Table XI shows such a large excess in the 
reliability coefficients of the judgments in comparison with those 
of the scores that any such analysis would of necessity be mainly 
determined by the judgments, and the weightings of the 16 vari- 
ables to yield components would be affected in a rather obscure 
manner by the differences in reliability of the two sorts of meas- 
ures. Accordingly, instead of analyzing the coefficients of Table 
XI, we have corrected all the coefficients of this table for attenua- 
tion giving Table XII, and have analyzed these relationships. 
This again is not very satisfactory because correction for attenu- 
ation when initial reliabilities lie between .25 and .43, as is the 
case with the test scores, introduces a chance element which it is 
impossible to eliminate, or, with any formula at hand, to estimate. 
We are thus, even with this 16 variable problem, forced to restrict 
our positive conclusions to things revealed by the first two or 
three components only. 

Remembering that based upon an earlier sample the scoring 
scheme of each response item for each trait was determined by the 
consensus of the judgments upon the pupil for the same trait, we 
should not anticipate any difference except that of reliability 
between Courtesy as judged and Courtesy as scored. We should, 
therefore, expect the weights for variables 1 and 11, 2 and 12, 
• . . 8 and 18 to correlate very highly for component I, again for 
component II, etc. In fact we find this to be the case for com- 
ponent I, as is clear from a study of Table XVII and Chart XIII, 
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but it is not the case for component II, the reasons for which we 
will shortly consider. Component I is apparently Puritania and 
nothing else. There is nothing of the unexpected in this finding 
and we will immediately pass on to a consideration of component 
II. 

This component from 16 variables and the sample of 511 bears 
no clear relationship to component II of any of the four preceding 
analyses. This lack of agreement is altogether surprising and calls 
for careful investigation. It being impossible to investigate it at 
present by experimental means, we must search for logical reasons 
for the discrepancy. Component 11 is obtained by weighting posi- 
tively all the judgments and negatively all the scores with one 
reversal, that in the case of Mastery. Now clearly there can be 
nothing in the psychology of children which would lead to such 
an outcome. Children can scarcely be courteous when being judged 
and discourteous when being tested by the association test. The 
cause must lie somewhere in the process of recording measures. 
Further, all judgment scores involved in the intercorrelations be- 
tween judgments are identical with the same scores when involved 
in the correlations with the association test scores, so we should 
not look for the cause of the peculiarity in any of the intercorrela- 
tions. The only measures that are left to be considered are the 
reliability coefficients, and here we may note that there may 
readily be an influence which may affect judgment reliabilities 
quite differently from test score reliabilities. If the half scores 
which led to the reliability coefficients for the judgments correlate 
spuriously highly, there would be no carry over to the test score 
reliability coefficients, and similarly no systematic error in the test 
score reliability coefficients would carry over to the judgment 
reliabilities. No large source of error in the reliability coefficients 
of the test scores suggests itself, but such a source for the judg- 
ments is immediately su^ested to one familiar with the mental 
labor, uncertainty, and^ sense of needing support which charac- 
terize the process of passing judgment as to traits of fellow beings. 
The judge, and we need not say the incompetent judge, faced with 
the request to appraise a fellow being along a line with which he 
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supposedly is familiar, will commonly satisfy the task by accepting 
hearsay, rumor, or reputation. The second judge does the same 
thing and in doing it accepts the same hearsay, rumor, and repu- 
tation. Thus there comes to be high agreement between judges, 
but low agreement with the trait actually possessed by the fellow 
being. The tendency is a commonplace experience in adult circles. 
Who has not been guilty of passing on an appraisal that is in no 
sense independent? Surely children are equally likely to form likes 
and dislikes and to make judgments of fellow classmates upon the 
basis of hearsay and not of independent knowledge. Though we 
consider this inevitable where independent knowledge is lacking, 
nevertheless it results in the making of judgments which are not 
independent in the sense demanded if a true reliability coefficient 
is to be determined. 

The writer early in this study was surprised but gratified at the 
high reliability coefficients found for the judgment measures. He 
now concludes that these are untrue, being spuriously high, and 
that the excess of the measured judgment reliability coefficient 
over its real reliability plus a certain chance element is the main 
cause of the second component as found in the case of the 16 
variable problem. A second, though less important cause, is appar- 
ently Rugged Individualism as shown by the weightings of Mas- 
tery. Rugged Individualism is not a large enough factor to domi- 
nate or determine the direction of the second component axis, but 
it is large enough to show its influence here, and, because not fully 
satisfied here, again in later components. 

An important consequence of the conclusion that the obtained 
reliability coefficients are spuriously high is that no interpretation 
of this second component based upon the psychology of school 
children need be sought, since the component is to be interpreted 
as a function of the psychology of the judges. Further, the very 
practical conclusion is drawn that judgments are not nearly as 
reliable as had been estimated in connection with these data. We 
niay believe that in general judgments of fellow beings are not 
nearly as reliable as obtained reliability coefficients would suggest. 

If we are right in attributing this second component, which the 
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reader will note is very substantial in size, its variance being 5.52 
out of a total of 16, to collusion, however unintentional, between 
judges, we may roughly estimate what the reliability of these 
judgments would be if the influence of rumor and hearsay could be 
eliminated. 

If we estimate the variance of the second component which is 
not due to hearsay as 2.52, which in the light of the other com- 
ponents is probably too large rather than too small, there remains 
a variance of 3.00 to be attributed to hearsay. Dividing this 
equally among eight variables yields .375 per variable. Thus, for 
example, in the case of Courtesy in place of the diagonal value 

reliability as obtained 

1.00 (which equals have (1.00— .375) 

V.82I 

reliability as it should be 

or .625 (which equals 7 = ;= ), The “reliability 

V.821 VMI ^ ^ 

as it should be” to give this answer is .513. Roughly this relia- 
bility based upon a composite of teachers’ and pupils’ judgments 
is as reliable as the average of four teachers’ judgments. Using 
the Spearman-Brown formula to obtain the reliability of a single 
teacher’s judgment of Courtesy, we obtain . 21 , This is very low 
but not out of harmony with estimates of teacher ability to judge 
scholastic achievement, which estimates would indicate a relia- 
bility in judging “reading ability,’’ “arithmetical ability,’’ etc., of 
between .3 and .4. It is only necessary to assume that teachers 
can judge of reading and arithmetical ability somewhat more 
competently than they can of Courtesy, Mastery, Fair Play, etc. 
The^indicated low reliability of teachers’ judgments is a handicap 
to experimental study utilizing'them as criteria measures. It does, 
however, emphasize the need of objective and reliable measures of 
character traits, if it is possible to obtain them. The low reliabilities 
of the scores from the association test as given suffer less in com- 
pauiaon with teachers’ judgments than would be the case had 
teachers* judgments reliabilities of .4, .5, or . 6 . If an association 
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test can be built up and scored yielding reliabilities of character 
traits of better than .5 per grade group, it will definitely serve a 
need not now met by any measures and it can serve as a basis for 
Studies of character and for studies whose aim is the derivation 
of a still better measuring instrument. 

Only a word or two need be said about the third component 
from the 16 variable problem. If our interpretation is correct, the 
hearsay nature of the second component has disrupted the analysis 
if taken as reflecting psychological relationships in children. The 
third component, which we might anticipate would be Rugged 
Individualism, does show heavy weighting of Mastery as judged, 
but no correspondingly heavy weighting of Mastery as scored. 
Since component II expressing opposition in weights of traits as 
)udged from the same traits as scored is only partially expressed 
so far as Mastery is concerned (—.05 and .44 in contrast with, 
say, Honesty .83 and —-42), a part of the essential opposition 
seems to be carried over, so far as Mastery is concerned, to com- 
ponent III, and perhaps again to component IV, for an examina- 
tion of the residual standard deviation column indicates that 
Mastery as scored will be the main ingredient of this fourth com- 
ponent, and Mastery as judged a much smaller portion of it. Let 
us, therefore, make no further attempt to interpret the third, the 
fourth, or higher numbered components. The surprising feature 
of this 16 variable analysis is the great size of the second com- 
ponent and its probable cause in factors outside of the children 
being measured. 

The conclusion of this study, that two or three components 
contain practically all the information to be found in the original 
eight, may seem contrary to **common sense. ^ The situation in 
its simplest case can be reduced to two dimensions. Consider a 
right triangle with hypotenuse c and sides a and b. We have 
= The quantities a® and b® correspond to the variances 
of components and c® to the total variance. If c®= 1, and a = .95, 
we obtain a* = .9025 ; b® = .0975 ; and b = .3 1. Thus b has a standard 
deviation of sufficient size to be far from negligible. The com- 
ponent b may be a variable which quite justifies a separate label and 
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a separate concept. We can perhaps say that roughly its right to 
a separate concept is represented by the size of its standard 
deviation, b/c, but that its importance in explaining the gross 
situation is represented by the size of its variance, b* /c*. We may 
readily believe that Courtesy and Honesty are different, though 
believing at the same time that the importance of that in respect 
to which they are different is very small with respect to the im- 
portance of that with respect to which they are the same. Thinking 
in terms of components involves thinking in terms of (1) variances 
of (2) measures which are independent of each other. 

It is believed that both (1) and (2) will greatly contribute to 
psychological analysis, though it is not claimed or implied that 
there are not additional requirements and aids to sound thinking 
in this held. 

Summary of Results of this Study 

It may be expected that when we have measures such as Strong 
Interest scores derived from a single basic set of items, we will 
have large error correlations just as was found to be the case with 
the eight different scores on the association test here used. 
Human judgments of fellow beings may be expected to have, in 
addition to the commonly recognized halo effect, a large hearsay 
effect. The halo effect leads to spuriously high correlations be- 
tween judgments of supposedly different traits, while the hearsay 
effect leads .rto spuriously high correlations between different 
teachers’ judgments of the same trait. The indication is that, after 
allowance for this, the reliability of teachers’ judgments of charac- 
ter traits is very low. 

Two or three independent character trait measures are highly 
successful in giving the gist of the complex story told by as many 
as eight trait measures chosen originally because of their separate 
importances in connection with school situations. 

The first of these measures is Puritania (a coined term) and the 
second Rugged Individualism. 

The method of analysis into principal components, perfected 
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by Harold Hotelling, proves straightforward and powerful, and to 
offer promise of wide usefulness. 

The association test shows promise as a means of measuring 
character traits, but as thus far perfected, is entirely insufficient 
for satisfactory individual measurement. 

There is great need for objective and more accurate measures 
of character traits if education is to be concerned with their 
development or modification. 

APPENDIX A 

The Method of Removing the Effect of Correlat on 
Between Errors in the Association Test Scores 

Let X , = Xqo + e , let a , = 1 

andx, = Xa + e, and(r,= 1 

Let r'i 2 = the correlation between Xi and xj before correction 
for correlation between errors in Ci and e», and let ri* = the corre- 
lation after such correction. 

Let X, equal the score on the first half of the first test 

T 

X, equal the score on the second half of the first test 

T7 

Xj^ equal the score on the first half of the second test 

1 

X,, equal the score on the second half of the second test 

Vt 

Then X, — X, = e, — e, 

T il T 


t - x^ = e^ - e 

T II 1 n 


r(x - X.) (x - x„) 

T TT T nr 


1(6, - e,) (e, 

T rr T 




from which 


r' z= ff T 

11 VcqOu couf 


"L <roo<^«i ^ocje* "f" “t" ^eje, 
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Assuming 

G0«| 

0 

and ^ 00 .. 

= 0 




1 




Assuming 

^co.. = 

0 

and 


= 0 


a\ - a 


1 1 

and 




and since ffa, 

= ff.Vr, 


Kelley, 

, Star. [166] 

and 

ffa = a, Vr, 






1 

> 

II 

« 

and 

1 

Vl 

- r. 

then r'„ 

= Vr, 

r , 

+ VI™ 

~ VT 

— r 

. re,e. 

Assuming 

e = 

'•l 

T n 

= 0 and 

% % = 

0 



e 

= 

+ e.) (e. 

TT T 

+ V “ 

- 

T 

- ej 



and therefore 

“ e^) (e^ “ Vl - r, Vl - r, 

vrT vf; 


To determine r^ the correlation between scores having reliabilities 
ri and rj respectively which would have been obtained if there 
had been no correlation between errors, use the usual formula for 
the relation between true and obtained coefficients, which gives; 

Vr, Vr, = - r(x^-x^) (x^^- x^)Vl-r, Vl-r, 
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t'H EATING IN THE CLASSROOM. WITH EMPHASIS ON 
THE INFLUENCE OF FRIENDS 


A. M. Jordan 

University of North Carolina 

The problem of how to encourage children to form habits re- 
garded as socially desirable is an ever present one. Among these 
desirable habits is that of common honesty; ideally, individuals 
should be honest under any and all circumstances. In certain 
classroom situations, however, the lack of honesty is very evident. 
A study of cheating by children in the classroom and of the causes 
therefor seems likely to throw light on the larger problem. It is 
such an investigation that is here reported. 

The Procedure 

In carrying out our plans it seemed too slow a procedure to wait 
until children showed evidences of dishonesty in the ordinary class- 
room activities. Furthermore, to prove that children cheat in the 
usual tests in the classroom is always difficult and uncertain. It 
seemed better in this investigation, therefore, to make use of some 
of the more artificial techniques which had been developed by for- 
mer investigators in the field. ^ Among the techniques available that 
called the I. E. R. (Institute of Educational Research) seemed 
most suitable for our purposes.* 

The I. E. R. technique consists in administering two tests so 
similar in nature that, if a class should take the two tests on two 
mornings near in time to each other, the scores would be practically 
the same. When the first of this pair of similar tests is given, an 
answer sheet is placed underneath the test with the admonition 
to the child not to look at the answers until the time is up. When 
the test is finished, the child is asked to draw out his answer sheet 
from under his test and to correct the test he has just taken, 
placing a “c” before the correct answer and an “x" before the 

'See May and Hartshorne, Studies in Deceit, Book 1, 1928, pp. 28—103, for 
a discussion of these techniques. . j u l 

*We were given permission to mimeograph these tests by May and nartshorne. 
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wrong answer. Within a few days the second of this pair of tests 
is given in a manner differing from the foregoing in that in the 
first instance the teacher was absent from the classroom and the 
investigator turned his back, thus giving the children a chance to 
add, if they chose, scores they had not actually made. On the sec- 
ond occasion, no answers are given, and the teacher is in the room 
assisting the investigator to make sure that the children receive 
no assistance whatever. 

There are then, two situations presented to the children : (a) one 
in which they have an opportunity for cheating and (b) another 
in which there is no opportunity for cheating. Couple these facts 
with the fact that the two tests are equivalent forms and it is seen 
that the scores obtained in Situation A minus those earned in 
Situation B give a measure of cheating. There were three such 
sets of tests used, (1) arithmetic problems, (2) completion of sen- 
tences, and (3) information. 

In February and March of 1930 these three sets of tests were ad- 
ministered to the chil||tren in Grades IV, V, VI, and VII in one of 
our North Carolina cities. Nine grades were involved in the study; 
one fourth grade, four fifth grades, two sixth grades, and two 
seventh grades — 276 children in all. These grades were located in 
two schools with four in one school and five in the other. In 
March of the following year this testing program was repeated in 
order to get a measure of the change in cheating habits during the 
intervening period of one year. 

During this year several types of evidence were collected : 

(1) Censuses were taken of friends at three-month intervals, 
four censuses in all. 

(2) The investigator visited each of these classes from five to 
nine times to investigate the behavior of the children, both in the 
class and on the playground. Each visit was from two to three 
hours in length. The teacher of the class furnished the investi- 
gator with a seating plat^ by which each child could be easily lo- 
cated. Notes concerning the behavior of the children were entered 
immediately or shortly after the observation was made. Ratings 
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of coopcrativencss and of general good behavior were made of each 
child from these personal observations. 

(3) Ratings of the child’s cooperativencss and general classroom 
behavior were secured from each classroom teacher. 

(4) Ratings of cooperativeness and general behavior were also 
secured from the physical education teacher in four classes. 

(5) The Sims test of economic and cultural level was adminis- 
tered in December, 1930. This test consists of a series of c|uestions 
such as, “Does your family own an auto which is not a truck 
“Where do you regularly spend your summers?” “How many 
magazines are regularly taken in your home?” which tend to throw 
light on the socio-economic level of the individual questioned. 

(6) The occupation of the father or mother was kept as a record 
by the school. This information was entered into our records. 

(7) The scores of the Stanford Achievement Tests were also 
obtained. The school gives these tests each year as a partial indi- 
cation of the right of a child to progress to the next grade. 

(8) The grade location and the age of each child was recorded 

also. * 

With these facts collected the question naturally arises as to 
what was their influence upon or relation to the matter of cheating. 

Amount of Cheating on Tests 

Under the conditions of this test when every child has an oppor- 
tunity to cheat, there is likely to be much more cheating than 
might be anticipated. Table I sets forth the percentages of cheat- 
ing in each of the tests used, and for each grade. If we consider the 
cheating on the arithmetic test only, we sec that the percentage of 
pupils cheating varies from 43 per cent in Grade V B-1 of the first 
school to 85 per cent in Grade V B-1 of the second school. In 
School 1 there is a tendency for cheating to increase with the ad- 
vance of pupils through the grades. The percentages of cheating 
in arithmetic run in round numbers 62, 43, 73, 83. Note that 
Grade V B-1 has a percentage of cheating far below that of the 
other grades. In attempting an explanation the principal of this 
school confirmed a suspicion which the experimenter had also en- 
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tertained^ that this class was one of the most indolent groups of 
children found in the whole school system. On account of these 
traits of character these pupils were really not enough interested 
in the tests to want to cheat. On the other hand, VI B-2 was one 
of the most ambitious classes visited. The members of this class 
enjoyed test comparisons with other classes in any test or in any 
other form of competition because they surpassed all other classes 
in almost every comparison. One hypothesis which might be 

TABLE I 
SCHOOL I 

Amoukt of Cheating by Grade and on Each Test in the Two Schools Studies 
N** Number in Class. C=Number Cheating. P™ Percent Cheating. 


Grade IV. B-3 Grade V, B-1 


First Year; Arith. Comp. Inf. Arith. Comp. Inf, 

N 34 36 36 28 29 29 

C 21 11 6 11 10 9 

P 61.8 30.6 16.7 42.9 34.5 31.0 


Grade V, B-2 Grade VI. B-2 


N 33 34 34 29 28 28 

C 24 16 12 24 5 11 

P 72.7 47.1 35.3 82.8 17.9 39.3 


Grade IV, B-3 Grade V, B-1 

Second Year: « 


N 30 30 30 19 20 20 

C 16 4 S 13 13 13 

P 53.3 13.3 16.7 68.4 65.0 65.0 



Grade V, B-2 


31 V 

30 

30 

22 

5 

6 


16.7 
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TABLE I — Contimud 
SCHOOL II 


Grade V, B-1 Grade V, B-2 Grade VI, B-1 

First Year: Arith. Comp. Inf. Arith. Comp. Inf. Arith. Comp. Inf. 


N 27 33 33 3S 36 36 34 33 33 

C 23 22 23 25 22 23 22 18 17 

p' 85. 2 66.7 69.7 71.4 61.1 69.4 64.7 54. S Sl.S 


Grade VII, B-1 Grade VH, B-2 


N 28 26 26 26 27 26 

C 19 9 6 16 12 8 

p 67.9 34.6 23.1 61.5 44.4 30.8 


Grade V. B-1 Grade V, B-2 Grade VI, B-1 

Second Year: • • ■ ' ’ 


N 21 20 21 22 24 23 20 18 18 

C IS 11 14 16 13 14 8 6 4 

P 71.4 S5.0 66,7 72.7 54.2 60.9 40.0 33.3 22.2 


Grade VII, B-1 Grade VII, B-2 


N 20 19 17 17 15 17 

C 17 8 7 10 3 9 

P 85.0 42.1 41.2 58.8 20.0 52.9 


Comparative Findings by May and Hartshorne* 


Public School (Suburban community, 
10 , 000 ). . . . * „ 
Per cent cheaung, population A, all 

tests 

Public School (Mid-western city, 200,000) 
Per cent cheating, population B, all 

tests 

Public School (metropolitan area-above 
average). 

Per cent cheating, population C, all 
tests 


V, 

B-1 

V, 

B-2 

VI, 

B-1 

VI, 

B-2 

VII, 

B-1 

VII. 

B-2 

75 

69 

66 

44 

21 

34 

41 

57 

25 

37 

31 

48 

54 

56 

54 

64 

66 

59 


(J) Miy and Hartihorne, Srudir/ «'« Boot 2, p. 54 ff. 
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offered is that the amount of cheating may he related to the urge 
present. In one case, one might hypothesize, with ambition low 
and the urge weak, the results are 43 per cent of cheating; in the 
other, with ambition overpowering and the urge strong, the re- 
sults are 83 per cent of cheating. In School 11 there is a steady 
decrease in cheating beyond the fifth grade. 

During the period of a year there were many changes among the 
responses to these tests. Referring again to Table 1, one observes 
that in the test of the second year in School 1 there is a decrease of 
cheating in the advanced section of the fourth grade in arithmetic 
and completion tests; the percentage on the information test re- 
mains the same over the one-year period. In Grade V B-1, there 
is an increase in cheating of nearly 100 per cent in all three tests. 
In School II changes in a year's time may be observed in five 
grades. In Grade V B-1, there are substantial decreases of cheating 
in arithmetic and completion, but only a slight change in informa- 
tion tests. Grade V B-2 shows only slight changes, while Grade 
VB-1 shows a very substantial decrease in all three tests. Note that 
the population of this grade has been reduced more than the aver- 
age, a fact which may account for the change. The lower seventh 
shows a definite increase in cheating, while the upper seventh de- 
creased in two instances and increased in the information test. 

Some of these changes are due to the fact that slightly different 
populations are compared. All the pupils in a grade were given the 
first test. A year later most of the pupils had been promoted but 
some had remained in the same grade and still others had shifted 
to different sections. This left a reduced number of those who 
were first tested to be tested the second time. Let it be perfectly 
clear, however, that all records of the testing in the second year 
were secured from pupils who had already been tested the first 
year. It is for this reason that there is always a reduction in 
population tested the second year. 

How do th^ ' cheating records of this study compare with those 
of different populations? Table I presents comparative data se- 
lected from the studies of May and Hartshorne. It is clear that 
there is a wide variation in cheating among the public school popu- 
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lations selected for comparison. These data were collected by 
means of the I. E. R. technique also, but differed in the respect that 
one more test (vocabulary) was used than in the study here re- 
ported. This gave the pupils one additional chance to cheat. 
Then, too, the May and Hartshorne figures arc for those who had 
cheated in any of the four situations, while the North Carolina 
figures are cheating scores on single tests. It is thus clear that in 
the May and Hartshorne technique the scores would be slightly 
larger than in the present case. If, however, the comparison is 
made of cheating scores on arithmetic tests, the difference between 
the results of the two investigations is very slight. In the North 
Carolina study on five pages of names, with thirty-five names to a 
page, there were only six cheating on any tests who had not also 
cheated on arithmetic. Cheating scores on arithmetic, therefore, 
will be used to compare with those of the previous study. 

In Grade V B-1, we find 43 and 68 per cent of the present study 
cheating, percentages which may be compared with 75, 41, and 54 
per cent of the previous one. In grade V B-2, the scores are 71 and 
73 per cent cheating in the present study in comparison with 69, 
57, and 56 of the previous one. Thus of the two fifth grades the 
lower section is below, the advanced section above, those studied 
by May and Hartshorne. In Grade VI B-1 the 65 per cent of this 
investigation is to be compared with the 66, 25, and 54 per cents 
of the previous one. In VI B-2, these scores are 83 per cent in the 
first and 44, 37, and 64 per cent in the second. The sixth grades of 
the North Carolina schools thus with one exception make larger 
scores on cheating tests than do these other schools. Grade VII 
B-1 shows 68 per cent cheating, a larger number than the 21, 31, 
and 66 per cent of the previous investigation. In Grade VII B-2 
there are 61 per cent who cheat, which may be compared with 34, 
48, and 59 of the previous investigation. It seems clear that the 
cheating records in the present investigation show slightly less 
cheating than the May and Hartshorne records in Grade V,but 
considerably more in the upper grades. This greater amount in 
the upper grades looms still larger if the comparisons are made 
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with Populations A and B only, for with Population C there is a 
substantial increase of cheating in the upper grades. 

The results show that the cheating in the population studied in 
this investigation is slightly greater than among those of a similar 
nature investigated by May and Hartshome. 


TABLE II 

PBaCENTAGB CHEATING AT £aCB AgE 

C. “Number cheating. N. C. “Number not cheating. Per. “Per cent cheating. 


Age 


9 



10 



11 


12 



C. 

N. C. 

Per. 

C. 

N. C. 

Per. 

C. 

N. C. Per. 

C. 

N. C. 

Per. 

School I 

17 

8 

68 

m 



32 

9 78 

6 

2 

75 ' 

Age 

13 

14 

IS 



1 

3 

25 

4 


100 




Age 

9 

10 

11 1 

12 

School II 


1 

0 

18 

3 

86 


— ar 

28 

10 

74 

Age 

13 

||_||||g 

15 



'22 

12 

65 


4 

m 

D 



Age 


9 


1 

10 


1 

11 


12 


Total 

17 

9 

65 

48 

17 

74 

, 60 

14 81 

34 

12 

74 

Age 


13 



14 


■I 

m 





23 

IS 


19 

4 

83 

D 

nO 




May and Hartshome, age. .. 



9 

10 

11 

12 

13 

14 

IS 

Population A, Per 




• > • 

45 

41 

19 

36 

35 

42 

39 

Population B, Per 


— 



41 

32 

33 

34 

41 

44 


In Table II, the percentage of cheating is calculated for each 
age. The total scores undoubtedly give us the truer picture be- 
cause of the lltger numbers of cases included. Considering, there- 
fore, the total scores, there is evident a gradual rise in the percent- 
ages cheating until the Seventh year is reached. Ages twelve and 
thirteen show a dropping off with another rise at the years fourteen 
and fifteen. At age fifteen there are not enough cases from which 

(444J 






























relation of socio-economic status to cheating 

to make inferences, but what data there are indicate an increase of 
cheating at that age. We may say, in general, that there is little or 
no diminution of cheating with age but rather an increase. 

The results from thework of May and Hartshome^areso different 
from those here reported that they need considerable comment. 
In the first place, their results do not seem to be consistent. For 
example, in Grades V B-1, V B-2, and VI B-1, of Population A, the 
percentages of cheating for all pupils are 75, 69, and 66, respective- 
ly. The pupils most likely to occupy these grades are those of ages 
ten, eleven, twelve, and a few at age thirteen. But the percentage 
cheating at these ages are 41, 19, 36, and 35. All these figures are 
for Population A and were achieved on the I. E. R. school test. 
When all the members of the population are assigned to grade 
sections (reported as Grade V, Grades V and VI, or simply Grade 
VI, which were not included in the grade percentages mentioned 
above) the percentages run 75, 66, 50, and 44 for grades V B-1, 
V B-2, VI B-1, and VI B-2. Not a year of ages ten, eleven, twelve, 
and thirteen has a percentage as high as the lowest reported here. 
We have then the anomalous situation of the members of a grade 
cheating from 65 to 75 per cent, but the ages composing the 
grade cheating from 19 to 41 per cent. The computation for age 
was evidently not based on the same figures as that for grades. 
Taking these figures at face value, the pupils of the present in- 
vestigation age by age cheated a very great deal more than did 
those who were the subjects of the former investigation. 

Relation of Socio-economic Status to Cheating 

The Sims test of socio-economic status was given to all the 
pupils included in this study who were present on the day of 
testing. This test is a reliable test of cultural and economic status. 
Questions such as the following are asked the pupils: 

1. Do you take private lessons in music? 

2. How many magazines are regularly taken in your home? 

3. How many rooms does your family occupy? and how many 
persons occupy these rooms? 

'Op. cit., p. 441. 
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There are twenty-three such questions in all. The reliability of 
the test has been computed by correlating the even answers with 
the odd in which case the coefficient was .83; and also the scores 
of matched sibs, which gave a coefficient of .94. In either case the 
reliability is sufficiently high for our purposes. 

The question before us is the effect of the cultural and economic 
level of the family upon the child’s cheating. This problem was 
approached in two ways: (1) the number of points possible to 
score on the test was divided into three groups, 5-13, 14-26, and 
27-32, and the percentage of pupils cheating was computed for 
each group; (2) the coefficient of correlation was computed be- 
tween the scores received on the cheating tests and those on the 
Sims test. 

Table III shows the percentage of cheating at each level, for 
each grade, in both of the schools. Totals are also presented. 
From the consideration of these latter, it is clear at the first glance 
that, contrary to what one might expect, ‘ there is an increasing 
amount of cheating with increasing socio-economic level. In 
School I the percentage cheating at the lowest economic level 
is 69; at the highest 77. In School II the percentage cheating at 
the lowest level is 75, while 100 per cent in the highest economic 
level cheated. Among the grades studied those in School I who 
scored lowest (5-13) on the Sims test tend to increase; while in 
School II the tendency is in the opposite direction. In School I 
the middle group (14-26) also increases with the higher grades; 
while in School II there is no tendency in any direction. In School 
I the group scoring highest (27-32) increase in cheating up to 
Grade V, then decrease somewhat; in School II in this group there 
is 100 per cent, cheating in every case. 

‘May and Hartshorne found that there was a negative correlation between 
Sims socio-economic scores and cheating in school. They report, however, in a foot- 
note on page 214, “It might be noted that the r between Sims score and School XI 
is positive in School P, and only slightly negative in School C, yet when the two 
schools are combined in one 'plot the r becomes —.490.” It is then clear that in 
one of the schools studied the correlation between the Sims score and cheating was 
positive, while in the other school the r was negative. In the present study the r 
was almost nro in both cases. 
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TABLE III 

Percewtaoe of Cheating at Each Level of the Sims Socio-Economic Test 

SCHOOL I 


N.“ Number in class. C.- Number cheating. P.»Per cent cheating. 


Grade 

Sims Level 

N. 

c. 

P. 

IV. B-3 

5-13 

5 

2 

40.0 

14-26 

16 

10 

62.5 


27-32 

15 

li 

5S.J 


Total 

36 

20 

55.6 

\'. B-1 

5-13 

9 

7 

77. S 

14-26 

10 

5 

50.0 


27-32 

6 

6 

100.0 


Total 

25 

18 

72.0 

V, B'2 

S-13 

1 

1 

100.0 

14-26 

12 

9 

75.0 


27-32 

20 

17 

S5.0 


Total 

33 

27 

SI. 8 

V[, B-2 

5-13 

1 

1 

100.0 


14-26 

12 

11 

91.7 


27-32 

16 

13 

81.2 


Total 

29 

25 

86.2 

Grand Total 



123 

90 

73.2 

All Grades 

5-13 

16 

11 

69.0 


14-26 

50 

35 



27-32 

57 

44 

77.0 


SCHOOL II 


V, B-1... 

Grade 

Sims Level 

5-13 

14-26 

27-32 

Total 

N. 

15 

13 

1 

29 

C. 

14 

12 

1 

27 

P. 

93.3 

92.3 
100.0 

93.1 

V. B-2... 


5-13 

9 

8 

88.9 



14-26 

21 

19 

90.5 



27-32 

5 

5 

100.0 



Total 

35 

32 

91.4 

VI. B-1... 


5-13 

13 

8 

61.5 



14-26 

15 

11 

73.3 



27-32 

4 

4 

100.0 



Total 

32 

23 

71.9 
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TABLE lll—Contirnud 
SCHOOL II — Continued 


Grade 

VILA-1 

Sims Level 
5-13 
14-26 
27-32 
Total 

N. 

13 

9 

1 

23 

C. 

8 

9 

1 

18 

P. 

61.5 
100.0 
100.0 

78.5 

VII, A-2 

S-13 

6 

4 

66.7 


14-26 

13 

11 

84.6 


27-32 

3 

3 

100 0 


Total 

22 

18 

81.8 

Grand Total 

— 

141 

118 

83.7 

All Grades 

5-13 

56 

42 

75 


14-26 

71 

62 

87 


27-32 

14 

14 

100 

All Grades, Both Schools .... 

5-13 

72 

53 

74 


14-26 

121 

97 

76 


27-32 

72 

58 

81 


Total 

265 

208 

78.4 


TABLE IV 

PERCEm-AOE Cheating at Each Level or the Sims Socio-Economic Test 

Both Schools Combined 



• 

IV. 



V. 



VI. 



VII. 

Sims Level 

N. 

C. 

P. 

N. 

C. 

P. 

N. 

C. 

P. 

N. 

C. P. 

5-13 

. 5 

2 

40 

34 

30 

88 

14 

9 

64 

19 

12 65 

14-26 

. 16 

10 

62 

56 

45 

80 

27 

22 

81 

22 

20 91 

27-32 

. 15 

8 

53 

32 

29 

91 

20 

17 

85 

4 

4 100 


Table IV shows the combined results for both schools. In the 
group scoring from 5-13 ^on the Sims test Grade V has a slightly 
higher average on the tests of cheating. Among the group receiving 
14-26 on this test there is a steady increase of cheating through the 
seventh grade. Except in Grade IV, the 27-32 point group show 









RELATION OF SOCIO-ECONOMIC STATUS TO CHEATING 

almost universal cheating. There seems to be something in the 
school environment that causes cheating to increase. Furthermore, 
those of good social background seem to cheat as much as those 
of poor. 

'I ables V, VI, VII summarize in an analytical way the cheating 
scores received by pupils at each of the three levels on the Sims 
score card on socio-economic level. It must be remembered that 
the higher the score on the Sims score card, the higher is the eco- 
nomic level and that the larger the cheating scores, the greater 
is the amount of cheating. The tables of distribution at each grade 
enable one to observe directly the amount of scatter. For example, 
in School I, Level I, Grade IV, two very different groups of chil- 
dren may be seen. Out of fifteen children, eight evidently did not 
cheat at all, wLile seven cheated a great deal — as if they, once 
embarked on such a cheating career, were going to cheat as much 
as possible. None of the distributions is symmetrical. 

In School I there is a much larger proportion of pupils in Level 
I (27-32) than in School IL there being 55 in School I and only 13 
in School II. On the other hand, in School II there are 48 in Level 
III (5-13) while in School I there are only 12 in this level. There is 
not such a great difference in the second level. These facts give us 
added confidence in the test, for School I is attended by many 
children of bankers, physicians, professors, real estate agents, 
while many sons and daughters of operatives of textile and tobacco 
factories attend School II. When Sims levels are compared, it is 
found that cheating scores are higher in School 11 at all levels in 
all grades. One must distinguish sharply between cheating scores 
and the percentage cheating. Cheating scores refer to the amount 
of difference between two equivalent forms of a test, one given 
with answers; the other, without. The percentage cheating refers 
to the proportion of students out of the total number tested who 
avail themselves of the opportunity to cheat. Compare, if you will. 
Level I in the two schools. In School I, in Grade V B-2 the score 
is 37.50 (Table V); in School II, the corresponding score is 45 
(Table VI). With Level II, School II scores are higher wherever 
comparisons are possible; namely at Grades V B-1, and V -B2. As 
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between the levels, Table VII shows that in Grade IV, Level I 
shows the smallest cheating scores followed by Level III, with 
Level II a poor third. In Grade V, Level III has the lowest scores, 

TABLE V 

Crbatiko at Each Letbl or Scokbs Obtainbo om Sims Score Card 

SCHOOL I 


Level I Level II Level III 

Cheat- Cheat- ^ Cheat- . * - 

ing IV V V VI ing IV V V VI ing IV V V VI 
Scores B-3 B-1 B-2 B-2 Scores B-3 B-1 B-2 B-2 Scores B-3 B-1 B-2 B-2 


70 





70 





70 





65 

, , 


, , 


65 

1 


1 


65 





60 



• » 


60 





60 





ss 



5 

1 

55 

2 


, * 


55 





50 

1 


3 

1 

50 


1 

1 

1 

50 





45 

1 

2 



45 

1 




45 

1 



i 

40 

S 


1 

3 

40 

2 


3 

1 

40 





35 


2 

1 


35 

1 

2 


, 

35 





30 



2 

3 

30 

, , 



, 

30 





25 



1 

1 

25 

2 


2 

3 

25 

1 




20 



2 


20 


1 

, , 

1 

20 


3 



15 



, , 

1 

15 


, , 

1 

1 

15 


1 



10 

2 


, , 

1 

10 

1 

1 


2 

10 


1 



5 

2 



2 

5 

2 

1 

2 

1 

5 

1 

. , 



0 

2 

1 

3 

3 

0 

1 

2 


, , 

0 

1 

2 



-S 

1 


1 

* 

-5 

2 

2 

* , 

1 

-5 

. , 

. • 



-10 

1 




-10 

, * 

, , 

* , 


-10 

• . 

• , 



-IS 

, , 




-IS 

, , 

, , 

I 

, , 

-IS 

. , 

. . 



N 

15 

5 

19 

16 

N 

IS 

10 

11 

11 

N 

4 

7 


1 


Median Cheating Scores at Each Sims Level 


Grades 


Grades IV, B-3 V, B-I V, B-2 VI. B-2 


Uvd I (27-32). STtS iiTTS 37lo 30^ 

Levdn (14-26) 28.75 10.00 28.75 22.50 

Level III (5-13) 17.50 17.50 
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TABLE VI 

Cheating at Each Level of Scores Obtained on Sims Score Card 

SCHOOL II 




Level 1 



• 

Level II 



Cheating 

V 

V 

VI 

VII VII 

Cheating V V 

VI 

VII 

VII 

Scores 

B-1 

B-2 

B-l 

B-l B-2 

Scores B-l B-2 B-l 

B-l 

B-2 

»0 





80. 

... 1 



, 

75 





75. 

... 1 

^ , 



70 


, , 

, . 


70. 


1 .. 



65 





65. 

... i 

2 



60 


i 

i 

i 

60. 

... 3 

1 



55 


1 

1 


55. 





50 





50. 

... 2 

2 

3 


45 

i 




45. 

... 1 

2 2 

1 

2 

40 



i 


40, 

... 2 

2 2 


2 

35 




i 

35. 

1 I 


, , 

30 


i 



30. 

1 1 

i 

1 

25 


1 

i 


25. 


5 

1 

1 

20 





20. 

1 2 

1 

1 

15 





15. 


2 

1 

. 

10 




i 

10. 


1 


2 

5 




1 

5, 



• • • 

i 

2 

0 





0. 

1 1 


. , 

-s... 





-5. 



1 



-10. .. 





-10. 





N 

i 

^ 

4 

i 3 

N. 

11 18 15 

9 

11 





LEVEL III 





Scores. , . 

. . . . 



. V, B-l 

V, B-2 

VI, B-l 

VII, B-l 

VII 

,B-2 







Level III 




80.... 










75.... 




i 






70.... 




1 






65. . . . 




. 

i 

i 




60.... 




i 

1 





55.... 





1 


i 



SO.... 




i 

1 

i 




45 ... . 




2 

1 

1 

2 



40.... 





2 


, , 


2 

35, ... 






2 

, , 



30, ... 




i 


1 

1 



25 ... . 




1 

i 


3 



20. . . . 










IS.... 






2 



i 

10.... 




i 


1 

2 



5.... 




2 

i 


1 


1 

0.... 






2 

2 


1 

-5..., 







, , 













N... 




ii 

9 

li 

i2 


s 
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TABLE Vl—C<nainued 
SCHOOL II — Continued 
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RELATION or SOCIO-ECONOMIC STATUS TO CHEATING 

TABLE VII 
LEVEL III 


Scores 

't 

Grades 


Total 

IV 

V 

VI 

VII 

80 


.. 




75 


1 



1 

70 


1 

, , 


1 

65 


1 

1 


2 

60 


2 



2 

55 


1 


1 

2 

50 


2 

1 


3 

45 

1 

3 

2 

2 

8 

40 


2 


2 

4 

35 


, • 

2 

, , 

2 

30 


1 

1 

1 

3 

25 

1 

2 


3 

6 

20 


3 

, , 


3 

IS 


1 

2 

1 

4 

10 


2 

1 

2 

S 

5 

1 

3 

, , 

2 

6 

0 

1 

2 

2 

3 

8 

-5 

4 , , 4 • 





-10 

4.4 • • 

, , 


, , 


-IS 

. • . • . 

, , 

, , 

, , 


N 

4 

27 

12 

17 

60 



Median Cheating Scores at 

Each Levee 





Grades 








Total 


IV 

V 

VI 

VII 


Level I (27-32).. 

13.75 

39.17 

31.67 

25.00 

35.00 

Level II (14-26). 

28.75 

38.33 

28.33 

30.00 

31.87 

Level III (5-13). 

17.50 

32.50 

35.00 

25.83 

28.33 


followed by Level II and Level I in order. In Grade VI, Level II 
shows the lowest score, followed by Level I and Level III in order. 
Finally, in Grade VII, Level 1 shows the smallest score, followed 
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by Level III and Level II in order. As a whole, the largest scores 
fall at Level I, followed by Levels II and Levels III. This finding 
is diametrically opposite to that expected. 

Occupational Level and Cheating 

The relation between occupational level and cheating was also 
investigated. The school had a permanent record of the occupa- 
tion of each child’s parents. These records were referred to Taus- 
sig’s classification. Taussig divides occupations into five levels. 
“Group I Professional men, proprietors of large businesses, and 
higher executives. Group II. Commercial service, clerical service, 
large land owners, managerial service of a lower order than Group 
I, and business proprietors employing from five to ten men. 
Group III. Artisan proprietors, petty officials, printing trades 
employees, skilled laborers, with some managerial responsibility, 
shop owners and business proprietors employing one to five men. 
Group IV. Skilled laborers (with exception of printers) who work 
for someone else, building trades, transportation trades, manu- 
facturing trades involving skilled labor, personal service, small 
shop owners doing their own work. Group V. Unskilled laborers, 
common laborers, helpers, hands, peddlers, varied employment, 
venders, unemployed, unless it represents the leisured class or 
retired.”^ 

Tables VIII, IX, and X set forth the relation between the scores 
on the Taussig groups and cheating, first, school by school, and 
second, with the results combined. There are few general trends that 
can be recognized in the two schools taken separately. In School I 
the percentage of cheating tends to increase as one considers lower 
Taussig groupings; School II shows the opposite effect. When the 
results are combined there is a slight increase of cheating down to 
Group III, but a slight decrease in Groups IV and V. The summary 
table also shows a slight increase of cheating up to the sixth grade, 
although the fifth and sixth are separated by only one point. The 
greats amount of cheating appears in the fifth and sixth grades. 

^See Sims's modifications of Taussig's classification, Manual of Directions for 
At Sims Score Card for Socio-Economic Status, p. 9. 
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TABLE VIII 

Pbucentaob Crbatihg at Each Occupational Level 
SCHOOL I 

^•Number in class. C^Number cheating. P"> Percent cheating. 


Classes 

Taussig's Classification 

I II III 

IV 

V 

Total 

Grade IV, B-3 

N 

8 

14 

S 

9 


36 

C. . . 

4 

8 

5 

6 


23 

P 

50 0 

57 1 

100 0 

66.7 


63.9 

Grade V, B-l 

N 

3 

S 

8 

13 

1 

30 

C 

1 

1 

5 

10 

> > 

17 

P . . . 

33.3 

20 0 

62 5 

76 9 


56.7 

Grade V, B-2 

N . 

10 

13 

8 

4 

• ■ 

35 

C 

7 

8 

7 

3 


25 

P 

70 0 

61 5 

37.5 

75.0 


71.4 

Grade VI, B-2 

N 

.. .. 7 

8 

6 

8 

• • 

29 

C 

.. .. 6 

7 

5 

7 

• • 

25 

P 

... S5 7 

S7 5 

33.3 

37.5 


86.2 

Total N 

.... 28 

40 

27 

34 

1 

130 

Total C 

18 

24 

22 

26 


90 

Total P 

.. 64 3 

60 0 

81.5 

76.5 


69.2 
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TABLE IX 

Pekckktaoe Creating at Each Occupational Level 
SCHOOL II 


Tauuig's Oassiiication 


Clauei 

I 

II 

III 

IV 

V 

Total 

Grade V, B-1 

N 

j • « . « 

5 

9 

17 

1 

32 

C 

. . . • . • 

4 

7 

IS 

1 

27 

P 



80.0 

77.8 

88.2 

100.0 

84 4 

Grade V, B-2 

N 

2 

11 

13 

6 

6 

38 

C 

2 

10 

11 

5 

6 

34 

P 

... 100.0 

90.9 

84.6 

83.3 

100.0 

89.5 

Grade VI, B-1 

N 

3 

8 

7 

7 

10 

35 

C 

2 

8 

4 

3 

7 

24 

P 

... 66.7 

100.0 

57.1 

42.9 

70.0 

68.6 


Grade VII, B-1 

N 

C 

P 

Grade VII. B-2 

N 

C 

P 


Each Class 


Total N 

8 V 

29 

41 

52 

27 

157 

Total C 

7 

24 

31 

37 

20 

119 

Total P 

87.5 

82.8 

75.6 

71.2 

74.1 

75.8 


1 4 

1 2 

m.o 50.0 

2 I 
2 

100.0 


5 11 

4 8 

80.0 72.7 


7 11 

5 6 

71.3 54.5 


5 26 

3 18 

(SO.O 69.2 


5 26 

3 16 

60.0 61.5 
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RELIABILITY AND VALIDITY OF THE TESTS USED 
TABLE X 

Taussio’s Occupational Classification and Cheating 


Both Reboots Combined 

N.* Number in class. C."> Number cheating. P. Per cent cheating. 


Classes 

I 

Taussi^s’ 

Groups 

IV 

V 

Total 

Grade IV 

N 

8 

14 

S 

9 


36 

C 

4 

8 

5 

6 


23 

P 

... SO 

57 

100 

67 

. • 

64 

Grade V 

N 

... 15 

34 

38 

40 

8 

135 

C 

... 10 

23 

30 

33 

7 

103 

P 

... 67 

6S 

79 

83 

88 

76 

Grade VI 

N 

... 10 

16 

13 

15 

10 

64 

C 

8 

15 

9 

10 

7 

49 

P 

Grade VII 

... 00 

94 

69 

67 

70 

7? 

N 

3 

S 

12 

22 

10 

52 

C 

3 

2 

9 

14 

6 

34 

P 

... 100 

40 

75 

64 

60 

65 

Total 

N 

... 36 

69 

68 

86 

28 

287 

C 

... 25 

48 

53 

63 

20 

209 

P 

... <59 

70 

78 

73 

71 

<59 


Reliability and Validity of The Tests Used 

The tests used in the investigation here reported were chosen 
because their reliability and validity had already been carefully 
determined. At the same time, it seemed useful to know how valu- 
able they were with the population studied. Measures were at 
hand to compute both the reliability and the validity. 

A — Keliahiliiy of Intelligence Tests 

As measures of intelligence the battery of three tests used 
correlate as follows: 
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TABLE XI 

RELiABiLnr or iMTEUiaBHCE Scores (One ye» apart) 


Intel. 1 and Intel. II - . 742 (N, 167) 

Arith. (B) I and Arith. (B) II - . 763 183) 

Completion (B)I and Completion (B) II >- .702 (N, 185) 

Information (B) I and Information (B) II » .457 (N. 182) 


These coefficients computed between scores separated by an 
interval of a year are a measure both of the reliability of the tests 
when used as measures of intelligence and of the change which had 
taken place in the individual in intelligence in a year’s time. May 
and Hartshorne report a coefficient of .87, which indicates the 
reliability of the tests as measures of intelligence when these 
measures are taken close together. If this .87 is compared with the 
present .74 between measures a year apart, changes in the indi- 
viduals themselves are indicated. 

The reliability both of the batteries (represented by “Intel. I” 
and “Intel, 11") and of the individual tests is sufficient for our 
purposes. 

B — Validity of InuUigence Tests 

These tests of intelligence were correlated with grade placement 
as measured on the Stanford Achievement Test and with the rat- 
ings of cooperativeness made by the teachers and the investigator. 

TABLE XII 

Correlations of Intelligence Tests and Stanford 
Achievement Grade Placements 


Intel. I and Stanford Achievement. — .820 (N, 271) 

Arith. Reas. (B) I and Stanford Achievement .... * , 667 (N, 280) 

Completion (B) I and Stanford Achievement =>= .679 (N^, 280) 

Information (B) I and Stanford Achievement = .572 (N, 279) 

IntilliMnce II and Stanford Achievement -= .739 (N, 176) 

Arith. Reas. (B) II and Stanford Achievement — * .565 (N, 190) 

Completion (B) II and Stanford Achievement. . . . >» .590 (N, 189) 

Information (B) II and Stanford Achievement = .461 (N, 186) 

I»FirstTest. II * Second Test. 


This table shows hjgh correlations both with the test as a whole 
(IntelUgence I, and Intelligence II) and with the sub-tests as well. 
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It IS thus seen that this test of intelligence correlates more closely 
with the Stanford Achievement Test given close to it than with 
itself given a year earlier. 

One could hardly expect an intelligence test to correlate very 
highly with rating scores of cooperativeness because quite different 
traits are being measured. 


TABLE XIII 

Correlations op Intelligence and Ratings por Coops rati veness 
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The highest coefficient here is only .48, but, before condemning 
the test as lacking in reliability, let the following table be con- 
sidered : 


2. F alidity of Cfuating Tests. 

TABLE XV 

Intercorrelatioks of Tests of Cheating 


Cheating score Arithmetic with cheating score 

Completion * .610 (N, 201) 

Cheating score Information with cheating score 

Completion . 570 (N, 282) 

Cheating score Information with cheating score 

Arithmetic -.539 (N, 264) 


From these data it is clear that the elements of the test have 
“marked” correlations with each other. It may be inferred from this 
fact that the test, as a whole, measures cheating fairly well since 
there is such a consistency between the elements of the test. 
Precisely the same type of situation is presented to the children in 
the case of each test and hence intercorrelations between the sub- 
tests taken near together may be regarded as the better indicator 
of their reliability. The next section, as a whole, may be con- 
sidered in connection with the validity of these tests of cheating. 

The intercorrelations of the ratings in the present investigation 
are not much different in size from those found in other investi- 
gations. 

.The closest resemblance found is that coefficient between the 
rating of the investigator and that of the teacher of physical 
education (r = .4S6). 

TABLE XVI 

Intercorrelation of Ratings or Cooperativeness 


Experimenter’s ratings of cooperativeness with room teacher’s. . . = .412 (N, 235) 
Physical education teacher’s ratings of cooperativeness with room 

teacher’s. = . 354 (N, 136) 

Experimenter’s, ratings of cooperativeness with physical educa- 
tion teachers = .456 (N, 130) 
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Relation of Cheating to Other Variables 

A — Cheating Scores of Pupils’ Friends 

The relation between the cheating scores of children and those 
of their friends constitutes the central theme of this investigation. 
By means of censuses of pupils* friends taken at three-month 
intervals over a period of twelve months, it was hoped to gain 
some insight into the role that friends play in cheating. Coeffi- 
cients of correlation were consequently computed between the 
cheating scores of each individual and those of his four (or fewer) 
best friends. This made eight correlations in all because each pupil 
had two cheating scores, one on the first test, and another a year 
later. 

TABLE XVII 


Coefficients of Correlation Between Children’s Own Cheating Scores 
AND the Cheating Scores of Friends 



Census 

Census 

■nniH 

Census 


I 

n 


IV 

Own Score (1). . . . 

.30 (N,21S) 

.282 (N, 185) 

.048 (N, 155) .098 (N, 146) 

Own Score (2). . . . 

.203 (N, 146) 

. 161 (N, 128) 

.315 (N, 122) 

.306 (N, 136) 


“Own Score (1)’’ was given at the beginning of the year at which 
time the first census of friends was taken. The first (1), second 
(II), third (III), and fourth (IV) censuses of friends appeared at 
successively greater distances from the cheating test. The de- 
creasing correlations at Censuses II, III, and IV are accounted for 
by changing friends and changing scores in the same friends. The 
second cheating score. Own Score (2), correlates more highly with 
the third and fourth censuses which were near in time to it. The 
third and fourth censuses were taken within three months of the 
second cheating score, just as the first and second censuses were 
within three months of the first cheating score. These two lines of 
correlation are consistent in showing “present but low** correla- 
tions between individual cheating scores and present friends* and 
negligible correlations between individual cheating scores and 
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friends* scores taken 6-9 months in either direction. If the factor 
of age is partiaJed out from the relation between a pupil’s own 
cheating score and that of his friends, the .30 of the first census 
becomes .295, a negligible change. 

Similar results are obtained when correlations are computed 
between the cheating scores of the four censuses of friends. The 
average of the four friends’ cheating scores was used with each 
census. 

TABLE XVJIJ 

INTERCOILRELA.TIONS OR FrIEMDs’ CHEATING SCORES* 


II III IV 

I 32 (N, 190) .208 (N, 157) .158 (N, 148) 

II 331(N, 141) .249 (N, 128) 

III 567 (N, 133) 


Censuses near together in time, I and II; II and III; III and IV, 
thus have much higher coefficients than those more distant in 
time, I and III, I and IV, and II and IV. If there is any influence 
among friends in the matter of cheating, it comes from those 
friends who are contemporaneously associated and not from those 
removed by a short period of time. 

Another method which aids in interpreting the results is that 
of Pearson’s coefficient of mean square contingency. By means of 
this twofold classification there may be computed the difference 
between chance resemblance and that which is present. For 
example, in Table XIX there are 75 cases studied which showed 
during one year a definite change either toward or away from 
cheating. The question to be answered is, “Is there a tendency for 
friends to change pari passu and in the same direction as for the 
individuals concerned?” 

*Th«ie fiaduig*^ afree cloiely with the r of .35 of the M»y and Hartahorne inveMigation when 

£ (1100 between tem wai the tame at own tcorc with fint or second cheating icorea of friends, 
y and Hambome, Sttu/ies tn Dtfiit, p. 275. 
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RELATION OF CHEATING TO OTHER VARrABLES 

Table XIX shows that those students who increased their 
cheating from the first to the second test had friends little like 
themselves, for, out of 33 individuals who increased their cheating 
scores, only 17 sets of friends increased their scores, while 16 sets 
did not. Quite different results appear with those students who 
decreased^ their cheating, for their sets of friends decreased their 
cheating scores in 31 cases out of 42. The coefficient of contingency 
is somewhat higher than .267 indicates, since the highest possible 
coefficient in a twofold table is .707. The coefficient is likewise 
about .3 when correlation is computed between individuals and 
their closest friends’ scores. This coefficient indicates a resemblance 
between the cheating of individuals and their friends’ cheating 
scores. Quite Interesting is the fact that in fifty out of sixty-three 
cases friends cheated when the individuals cheated. 


TABLE XIX 

Chanced Scores in Individuals Compared with Changes in 
Friends’ Scores in Cheating 




Friends 


Individual 

Increase 

Decrease 

Total 

Increase 

(12) 

(21) 

33 


17 

16 


Decrease.. 

(16) 

(26) 

, , 


11 

31 

42 


28 

47 

75 


25 25 25 25 

X* — +-+-+- 
12 16 21 26 


Coefficient of Mem Square Contingency 


I 5.79 

C=J .267 

>75+5.79 

Highest possible C with 2-fold classification .707, 


[ 463 ] 



TEfiTS AND MEASUREMENTS 


TABLE X\X—Continutd 


Individual'* Cheating Scoke with That of Best Friend 


Individual's Score: 

N 

Friends 

C 


N 

(10) 

(23) 



17 

16 

3.3 

C 

(20) 

(43) 

63 


13 

SO 



30 

66 

96 


N -Not cheating. C = Cheating. 


Coefficient of Mean Square Contingency. 

49 49 49 49 
X‘=— +— H — H — =10.55 
10 20 23 43 


I 10.55 

^.31 

96+10.55 


Correlations between Cheating Scores of an Individual and those of his best friend 
or friends show a present but low correlation. 

Correlations 


Individual cheating scores and those of his best friend = .194 (N, 192) 

Individual cheating scores and those of his 4 best friends = .172 (N, 134) 


May and Hartshorne report an r of .159 between the cheating scores of an individual and those 
of hia friend, when the friend was at school in another classroom. 


B-^keating Scores Compared with Ratings of Cooperativeness 

Ratings of cooperativeness were obtained from the physical 
education teacher, the room teacher, and the investigator. In 
general, we should expect that those pupils who cooperated the 
most in promulgating the activities of the classroom or playground 
would score less on cheating than those lacking in this character- 
istic. And so it turns out. The negative coefficients indicate clearly 
that there is ^an inverse ratio between cheating and cooperative- 
ness. The coefficient is not large, but is consistent in each case. 
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TABLE XX 

COMLELATIOMS BeTWKEN ChEATIHC ScOREt AND RaTINQS OF 

Cooperativeness 


Cheating scores (all) I test and ratings (EMerimenter) ■ — .117 (N, 210) 

Cheating scores (all) II test and Ratings (Experimenter) ■ — .212 (N, 139) 

Cheating scores (all) I test and Ratings (Classroom teacher). . - — .061 (N, 266) 

Cheating scares (all) II test and Ratings (Qassroom teacher) -- .162 (N, 158) 

Cheating scores (all) I test and Ratings (Playground teacher) - — .010 (N, 120) 

Cheating scores (all) II test and Ratings (Playground teacher) ■« — .406 - (N, 74) 


The sub-tests show similar results. 

Correlations Between Cheating Scores and Ratings of 
Cooperativeness 


Arith. (cheating score I) and cooperativeness (Experimenter’s 

ratings) - - .090 (N, 120) 

Completion (cheating score I) and cooperativeness (Experi- 

menter’s ratings) ■>-.20 (N, 122) 

Information (cheating score I) and cooperativeness (Experi- 
menter’s ratings) ■ — .186 (N, 122) 


Finally, when intelligence was made constant by the partial 
correlation technique, r between the experimenter’s ratings and 
cheating scores was reduced from — .20 to — .11. 

C — Cheating and Intelligence 

Our results show a negative correlation of a more substantial 
nature between cheating and intelligence. In one case a coefficient 
is as high as — .44. 

TABLE XXI 

Correlations Between Intelligence and Cheating 


Intelligence (B scores) I with cheating on 1st test. . 
Intelligence (B scores) I with cheating on 2nd test. 
Intelligence (B scores) II with cheating on 1st test. 
Intelligence (B scores) II with cheating on 2nd test 
Completion (B scores) I with cheating on 1st test.. 
Completion (B scores) I with cheating on 2nd test , 
Information (B scores) I with cheating on 1st test. 
Information (B scores) I with cheating on 2nd test. 
Arithmetic (B scores) I with cheating on 1st test. . 
Arithmetic (B scores) I with cheating on 2nd test.. 
I = First Test. 11 = Second Test. 


--.392 (N, 264) 
--.295 (N, 152) 
--.192 (N, 159) 
--.439 (N, 158) 
= -.337 (N, 264) 
--.340 N, 154) 
--.259 (N, 264) 

.247 (N. 152) 

--.178 (N. 264) 
--.184 (N, ISS) 
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Here, once more, the results are consistent. Note that the 
correlations are the same whether the tests as a whole are included 
or whether only sub-tests are used. The less intelligent cheat more, 
and the more intelligent less, than the average. The results from 
May and Hartshorne’s study are in general agreement with those 
here reported. Their coefficients between intelligence and cheating 
range from — .33 to — .49. 

A part of these negative correlations might be attributable to 
the relation of both intelligence and cheating to the Sims score of 
socio-economic level. This last factor, therefore, was made con- 
stant by the partial correlation technique. The correlation between 
cheating and intelligence with Sims constant is — .41. Thus the 
— .39 coefficient between intelligence and cheating is increased 
to — .41 when the effect of the Sims scores was teased out. 

D — Cheating and Age 

From Table XXII it may be deduced that there is a slight in- 
crease of cheating with age. Except in one case, all coefficients are 
small but positive. Age and cheating scores on the second test 
show a present, but low, coefficient. 


TABLE XXII 

CoRRILATIONS BETWEEN AgE AND CHEATING ScORES 


Age and cheating scores (ail) I 

Age and cheating scores II 

Age and cheating scores (Arithmetic) I 

Ag? and cheating scores (Arithmetic) II 

Age and cheating scores (Completion) I 

Age and cheating scores (Completion) II 

Age and cheating scores (Information) I 

Age and cheating scores (Information) II 

I — First Test, II — Second Test — one year later than the first. 


= .077 (N, 264) 
= .222 (N, 158) 
= .004 (N, 274) 
-.134 (N, 159) 
-.159 (N, 281) 
-.352 (N, 175) 
= .058 (N, 281) 
-.142 (N, 176) 


Age is thus not closely related to the factor of cheating in 
Grades IV to VII, inclusive.* 

■May and Hartshorne report coefficients ranging from —.025 to .289 with a 
etnnpoaite of -t-,146 (our average is .130). 
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The correlation between cheating scores and age, with the scores 
on the Stanford Achievement Test made constant, resulted in a 
change from — .004 to .019, a negligible one. 

E — Cheating and Stanford Achievement Test Scores 

Our data show a present, but low negative, correlation between 
cheating scores and the grade placement as computed from the 
Stanford Achievement Test. To eliminate the factor of age to a 
very large extent, correlations were computed from subjects all of 
whom were in grades IV and V. It will be noted that the coefficient 
is slightly lowered under this treatment (from — .248 to — .118), 
but in the case of the second test the coefficient increases from 
— .248 to — .355. Those boys and girls who score high on their 
daily work are less likely to cheat than those who score low. Con- 
trariwise, those low in their daily work are more likely to cheat 
than the others. 

When the relation between the Sims score and cheating on the 
one hand and the Stanford Achievement score on the other is 
eliminated, the correlation of — .25 between cheating and the Stan- 
ford Achievement score becomes — .277 or — .28. 

TABLE XXIII 

Correlations Between Cheating Scores and Scores on the Stanford 

Achievement Test 


Stanford and cheating scores I » — .248 (N, 256) 

Stanford and cheating scores II — — ,314 (N, 158) 

Stanford and cheating scores I (Grades IV, V only) = — . 118 (N, 149) 

Stanford and cheating scores II (Grades IV, V only) “ — .355 (N, 111) 

Stanford Achievement and Age = — .089 (N, 289) 


A very surprising effect is produced by partialing out the factor 
of intelligence from these relations. The correlation between 
Stanford and cheating scores, with intelligence constant, changes 
from — .25 to +.14, showing that this negative resemblance is 
due simply to the factor of intelligence, which correlates so highly 
with Stanford Achievement scores. 

[ 467 ] 




TESTS AND MEASUREMENTS 


In the May and Hartshorne study there is no direct comparison 
which can be made but only an indirect one. They report an r of 
— .20 and — .02 between scholastic averages and cheating scores, 
coefficients slightly less significant than the ones reported in the 
present study between the Stanford Achievement test and cheating 
scores. 


F — Cheating and Sims Scores 

Finally, we come to the relation between socio-economic level 
and cheating. 

TABLE XXIV 

Correlations Between Sims Scores and Cheating 


Suns and cheating I 
Suns and cheating II 
Suns and Arithmetic (cheating) I 
Sims and Completion (cheating) I 
Sims and Information (cheating) II 
Sims and Arithmetic (cheating) II 
Sims and Completion (cheating) II 
Sims and Information (cheating) I 
Sims and Stanford Achievement 
Sims and Intelligence I 


0S3 (N, 238) 
129 (N, 145) 
157 (N, 243) 
056 (N, 254) 
047 

051 (N, 161) 
164 (N, 159) 
014 (N, 254) 
286 (N, 265) 
237 (N, 251) 


These correlations range from a small positive relationship to a 
small negative relationship in such a way as to be convincing that 
in the present study there is little correlation between cheating 
scores and socio-economic level as measured in the Sims test. 

This argument is fortified by the results obtained by separating 
(tat the common element of intelligence from this relationship. 
The coefficient of correlation between the Sims scores and cheating 
scores is raised from .05 to .156 when the factor of intelligence is 
made constant. There seems to be little, if any, relation between 
socio-economic level and cheating. 

These results may be compared with a negative relationship of 
— .49 found by May and Hartshorne. They do say, however, that 
in one school the correlation is positive with cheating in school, 
but when die results were combined the negative relation results. 
On the other hand, they find little or no resemblance between 
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Sims scores and cheating scores on athletic and party tests of 
cheating. 

Summary and Conclusion 

Tests of cheating were administered to 275 pupils in nine dif- 
ferent gfades. The tests used were those developed by the Insti- 
tute of Educational Research (Teachers College, Columbia) and 
modified by May and Hartshorne for use in their investigation. 
They were administered twice, with a year between tests. During 
this year (1) four censuses were made of the friends of the subjects, 
(2) visits to the class and playground were made by the investi- 
gator, (3) ratings of cooperativeness and general class behavior 
were obtained from the classroom teacher, the teacher of physical 
education, and the investigator, (4) the Sims test of socio-economic 
level was administered to all, (5) the occupation of the father and 
mother of the children was obtained from the school record, (6) 
scores of the Stanford Achievement Test were entered on the 
records, (7) and finally, the grade, location, and age of each child 
were entered on the records. Correlations were then computed 
between the different variables assembled, and the influence of 
certain variables were partialed out whenever such a procedure 
seemed at all promising. 

(1) The amount of cheating under the conditions of the test 
was unexpectedly large. On one test of arithmetic reasoning the 
percentage of those cheating ranged from 43 to 85 per cent. There 
was a slight Increase in the upper grades. In comparison with the 
results of May and Hartshorne, the figures are closely similar in 
Grades V B-1 and V B-2, but in Grades VI and VII there was 
much more cheating among the pupils in this investigation. 

(2) Substantial changes in cheating scores appeared during the 
period of a year. The score in cheating is not nearly as steady as 
an Intelligence Quotient. The reliability coefficient, which ap- 
proaches ,75 when computation is made between tests that have 
been given near together, is reduced to .48 when the tests are given 
a year apart. Furthermore, 75 cases were found who changed defi- 
nitely during the year, either from “cheating” to “not cheating” 
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or vice versa. Out of 75 definite changes discovered, 33 increased 
and 42 decreased in their cheating scores. 

(3) Age and cheating are not substantially related. There is a 
slight increase in cheating from ages nine to eleven, then a decrease 
from twelve to thirteen, and finally another increase at fourteen 
and fifteen. The correlations between age and cheating range from 
.004 to .352 with an average of .130. This agrees substantially with 
the f of .146 obtained by May and Hartshorne. 

(4) Little relation was found between socio-economic level as 
measured on the Sims score card and cheating. Pupils with excel- 
lent economic opportunities were just as likely to cheat as those 
with meager opportunities. The correlations between the Sims 
scores and cheating scores ranged from — .056 to .157 with an 
average of .002. Nor was there any relation shown when the socio- 
economic level was determined from the occupation of the parents 
as entered on the school record. When these occupations were 
classified into the five Taussig levels, the percentage cheating at 
each level beginning with the highest was 69, 70, 78, 73, 71. These 
results agree with the correlations between the Sims test and 
cheating scores. These results are different from those discovered 
by May and Hartshorne, who report a coefficient of — .49, although 
they report that of two schools studied one showed a positive and 
the other a negative correlation. Moreover, in their study no re- 
semblance was discovered between the Sims scores and cheating 
scores on athletic and party tests. 

(5) Some interesting measures of reliability and of other rela- 
tionships were discovered in the present investigation. 

a. As measures of intelligence the 1. £. R. tests showed a self- 
correlation of .74 when the tests were given a year apart. 

b. As measures of cheating these tests correlated .48 a year apart. 
This is not a true reliability because of substantial changes in 
cheating. 

c. These I. E. R. tests as measures of intelligence correlated .82 
with Stanford Achievement grade level. 

d. As measures'of intelligence, these tests correlated from .17 to .42 
with teachers’ and investigators’ ratings of cooperativeness. 
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The inter-correlations of these ratings ran from .35 to .46. 

(6) Cheating scores of individuals and of their friends correlated 
in a very significant way. The censuses were taken four times 
during the year at intervals of three months. The first test of 
cheating was given near the first two censuses; the second 
test of cheating near the second two censuses. The r between 
individual and friends’ cheating scores for the first census was .30; 
for the fourth, .098. The coefficient between the individual’s cheat- 
ing scores on the second test and friends’ scores of the first census 
was .20; of the last census, .31. There is thus a clear tendency 
toward greater resemblance in those scores taken near in time. 
When age is partialed out, the results are the same. Moreover, 
when the coefficient of mean square contingency was computed 
between the changes in cheating between 75 Individuals and their 
friends the correlation was .267 (maximum twofold classification 
is .707), And finally, when this same coefficient was computed 
between an individual’s cheating or not cheating and his best 
friend’s cheating or not cheating the r was ,31. This evidence sug- 
gests that there is a small but definite tendency for friends to influ- 
ence each other in cheating. 

(7) Cheating scores and ratings for cooperation show a negative 
correlation. The coefficients range from — .01 to — .41 with an 
average coefficient of — .161. 

(8) Intelligence and cheating show the highest negative correla- 
tions, The coefficients of correlation range between — -.18 and — .44 
with an average at — .286. Those of greater intelligence tend to 
cheat less than those of less intelligence and vice versa. May and 
Hartshorne found here an r of — .493. 

(9) Cheating also is negatively related to the Stanford Achieve- 
ment Test. The coefficients in this case ranged from to 

— ,355 with an average of —.259. There is thus a slight telHlency 
for those who are well advanced in their work to cheat less and 
vice versa. 
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Chapter VI 


SUMMARY AND CONCLUSION 
A. C. Krev 

The plan upon which the Commission conducted its Investigation 
set aside one major division of the work for the preparation and 
application of new-type tests. The preceding pages have presented 
a fairly complete record of the work of this division, more or less 
in the order in which the work was done. The conclusions to be 
derived from this narrative of experience in the construction and 
use of new-type tests in the field of the social science subjects are 
probably already clear to those who have read these pages with 
care. It is a record of high hopes, great difficulties, many disap- 
pointments, and some achievements. These are here briefly sum- 
marized. 

In retrospect, there seems to have been little ground for the ex- 
tremely high hopes which had been entertained by the Committee 
which planned this division of the Investigation. The seven years 
which have intervened since that plan was drawn have witnessed 
great advance in the knowledge of the new-type test, with which 
at that time few, if any, of the social scientists had had much ex- 
perience, They were, therefore, disposed to accept the great claims 
made for this new instrument of research not only by those who 
had devised the instrument, but also by some scholars in other sub- 
ject jnatter fields who had found it of great utility. Few, if any, 
of those who had developed the technique of the new-type test 
were scholars of the social sciences as well. They therefore had no 
basis for assuming that the new-type test could not be applied with 
as much success to the special field of the social sciences as else- 
where. Both social scientists and experts in educational measure- 
ment were, therefore, in a frame of mind which could only result in 
considerable disappointment as their joint work progressed — as 
social scientistN learned more about the new-type test and the 
technician in measurement learned more about social science. 
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Statement of Objectives 

Those technically interested in educational measurement desired 
from the social scientists a statement of ends, aims^ and purposes 
of instruction in terms of objective knowledge, confident that once 
these objectives were so stated they could be definitely measured. ' 
Analysis of the materials and aims of social science instruction, 
however, indicated that no such simple solution of the problem of 
objectives was possible. The aim of social science was concerned 
not only with the acquisition of information or knowledge, but as 
much, or more, with the use of that knowledge in dealing with 
social matters to be faced by the pupil in adult life. The inclusion 
of such considerations of probability, possibility, and uncertainty 
introduced elements which seemed to defy complete objective state" 
ment. Furthermore, even the treatment of past knowledge, pre- 
sumably completed and therefore apparently capable of objective 
statement, was found to involve questions of judgment. As Beard 
has pointed outS whether it was a matter of accepting past knowl- 
edge or of applying knowledge to present or impending social 
situations, certain ethical and esthetic considerations were nearly 
always involved. The individual’s ideas of right and wrong, of ex- 
pediency or inexpediency, and his individual likes and dislikes played 
an important part both in his acceptance of knowledge of the past 
and in his use of knowledge for immediate needs. Knowledge of a 
purely objective nature alone did not determine conduct. It might 
condition conduct, and usually did so, but it was still possible for 
individuals possessed of the same knowledge to adopt quite differ- 
ent lines of conduct in a same social situation. Nearly every fact 
in social science was found to be the focus or nexus of a relation- 
ship which radiated out into the “seamless web’’ of world society. 
Nearly every idea was found to be part of a more or less extended 
pattern, which varied somewhat with each individual, and was 
more or less constantly changing. All this became increasingly ap- 
parent as work continued, and the actual trial of such tests as were 
constructed tended to confirm the above analysis. It was, there- 

‘Beard, C. A.; Nature of ike Social Sciences, vol. 7 in this geriei. 
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fore, impossible to state any major, and few minor, objectives of 
the social sciences in terms of objective or empirical knowledge. 
Such empirical data as social science employed represented but the 
beginning of social science, and the social scientist was primarily 
concerned with the subjective elements surrounding those data. 
At the risk of a play on words, it became clear that the most 
“objective” fact about social science was that it was so largely 
“subjective.” The clear recognition of these conditioning elements 
proved a source of great disappointment to technicians in measure- 
ment, for thereby existing tests were rendered practically useless, 
and a modification of existing techniques was forced — a process 
necessarily slow and doomed to very limited achievement in the 
time allotted to the Investigation. The disappointment was shared 
by the social scientists. 

Grade Placement 

Nearly every teacher of social science subjects feels that there is 
some natural order of progress in learning about society. It was 
hoped that the new-type test afforded the necessary instrument of 
research by which this order might be demonstrated. If so, the 
result promised would be of great aid to curriculum makers in this 
held, and equally so to teachers in determining their methods of 
instruction. “Grade placement" was the technical educational term 
applied, carrying obvious implications for both curriculum and 
method. Various efforts were made to discover the proper ap- 
proach for the construction of tests. 

Relative Difficulty of Courses in Social Science. The first approach 
attempted was to discover whether the several courses in social 
science presented inherent differences of difficulty. Examination 
of the numerous studies of curricula, published as well as unpub- 
lished, and of existing curricula in this and other countries failed 
to yield any decisive clues. The subjects generally employed in 
^hools as the basis of later instruction in social science were found 
to be history, geography, elementary political science, economics, 
and sociology in various forms. Little, if any, uniformity was found 
in the arrangement of these courses in the schools. A course placed 
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last in the series in one school system was found very early in an- 
other. Oriental history, taught as a rule in the advanced years of 
college in this country, was taught to small children in China and 
Japan, and even in a few schools of this country. Community 
study, which by its immediacy might promise to serve as the first 
stepping stone, and actually was so used in many countries, ap- ”■ 
peared likewise as an advanced course in universities of this and 
other countries. Even geography, which seemed completed by the 
sixth or seventh grade in most schools of this country, was regu- 
larly taught in the secondary schools of Europe as well and in 
colleges both here and abroad. The more analytical social sciences, 
politics, economics, and sociology, which are usually more abstract 
as well as more analytical than history or geography, generally 
appeared late in the secondary schools, but important exceptions 
were found even in this arrangement. The only safe inference to 
be drawn from these studies was that no school course in social 
science was inherently more difficult than any other, as such, with 
the possible exception that the more analytical social sciences, 
when taught in a more abstract form, might be so regarded. 

Relative Difficulty of Topics Within Courses. Having failed to 
find any decisive clues in the arrangement of courses, a more 
analytical study was made of the topics treated within courses. 
Analysis of courses of study had been a favorite theme for theses 
at most centers of educational research. Published and unpublished 
studies, the latter chiefly masters’ theses, were examined. Most of 
these analyzed courses by topics with a nucleus of time, place, or 
homogeneous activity. These studies were supplemented by ex- 
ploratory analyses of economic, political, sociological, and other 
elements.^ Differences in degree of detail, the specificity of detail, 
the use of imaginative language and of emotional qualification were 
also canvassed. It became clear, however, that, as in the case of 
courses, topics likewise could not be classified by title alone as in- 
herently of different order of difficulty. The Federal Reserve sys- 

‘Dorothy Bovee, Dorothy Houston, Margaretha Jorgenson, Wilbur Murra, 
Renata Pecinovski, Lena Van Bibber and Olga Wold, all teaeherg of some experi- 
ence, aided in this work. Miss Van Bibber’s study was published in the Second 
Yearbook of the National Council for Social Studies 1932. 
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tern of banking, reserved for intensive study in advanced courses of 
colleges and universities, was also treated in the elementary schools. 
If there were differences in difficulty as between topics, such diffi- 
culties lay in other aspects of the topic, not in the titles. 

Relative Dificulty of Terms Used in Social Science. Disappointed 
both in the arrangement of courses and the topics within courses, 
attention was reluctantly turned to the study of the vocabulary of 
social science. Whether the reluctance was due to a widespread 
feeling that the study of words belonged to the linguistic sciences 
and not to the social sciences, or to a vague abhorrence of separate 
words as expressions of social ideas and occurrences, was never en- 
tirely clear. The words, however, did seem to present a definite- 
ness and a precision which both the courses and topics had failed 
to yield. The existing word studies had been made for various 
purposes, linguistic, psychological, and educational. Some of those 
made out for spelling and reading contained ratings primarily on 
the basis of frequency of occurrence, though inferentially of some 
significance for relative difficulty also. There were also a few 
studies devoted to the special vocabulary of one or another of the 
social sciences. These included studies of concepts. The latter were 
found, upon analysis, to contain such complex matters as points of 
view, attitudes, and summary generalizations of large bodies of 
data concerning complicated problems. Much of this material of- 
fered the same difficulty found in the case of topics. Luella Pres- 
sey’s study of the special vocabulary of history, Woody and 
Stephenson’s list of terms in civics, and Eubank’s list in sociology 
were more specific. These pioneer efforts were supplemented by 
the work of Edith Ware, Frances Baldwin, and Edgar Wesley, all 
on the staff of the Investigation. It was found possible to make a 
general distinction between terms designating such definite mat- 
ters as persons, places, material objects, and events usually appear- 
ing as proper nouns and other terms which designated relationship. 
The latter appeared usually as common nouns or other parts of 
Speech. Their peculiar value lay in their more universal applica- 
bility. The members of the staff directed their efforts to the com- 
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pilation of such terms, as is indicated in the list contained In this 
volume.^ 

There were disappointments in the study of these separate terms 
as there had been in the study of courses and topics. The precision 
of single proved more illusory than real. Their definiteness 

tended to lie only in their spelling, and many of the terms were 
often used in a variety of meanings, and the same meaning was 
frequently expressed by a variety of terms. However, tests em- 
ploying the listed terms were constructed and tried out, with some 
interesting results. 

The Investigation was also aided in its study of grade place- 
ment by two scholars who were working independently on this 
problem. Professor Paul Lutz, of the Teachers College of Wash- 
ington, D. C., had been at work upon the problem for several 
years both here and abroad. Professor D. S. Brainard, of the 
Minnesota State Teachers College at St. Cloud, was at work upon 
a study of grade placement in American History. Both generously 
aided the investigation. Lutz also made available his translation 
of the essay by Seignobos on this subject. 

Natural Order of Learning in the Social Sciences 

Despite the numerous disappointments encountered in all this 
effort, the combination of analytical studies and tests yielded some 
results which may be of value. These findings are set forth tenta- 
tively, not through any lack of faith in their soundness, but be- 
cause the matters with which they deal are both so subtle and so 
complex that others may desire a greater accumulation of evidence 
than can be here presented. 

All matters of social concern have a location in place, and a 
setting in time, involving, of necessity, person and circumstance. 
Their relationship to other social occurrences, involves economic, 
political and cultural ideas. These ideas are expressions of social 
process in time, and yield yet further ideas applicable to future 
situations. The material elements are readily grasped, the other 
more slowly and in varying degree, according to the anteceden 
^See Appendix to this volume.. 
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knowledge, interest, and competence of each individual. Thus, the 
recent London Economic Conference, whose potential significance 
could have been appreciated by only a very few, highly trained in 
the fields of international politics and finance, presented certain 
elements which even the most casual reader of the newspaper 
grasped. Even pupils in the elementary schools could name the 
conference, locate it, and mention the leading nations involved, 
and some of the chief actors. 

The names of persons, places, and events are easily learned. 
Advance in knowledge consists in adding items of relationship. The 
degree of knowledge is measured, not by any particular item of 
relationship, but by the range of relationship or associations which 
the individual is able to connect with each name. 

Terms expressing ideas may also be easily learned. Advance in 
knowledge of such terms consists in recognizing the shades of 
meaning which adhere to the terms in their various connectic ''s. A 
simple definition such as ordinary dictionaries afford usually repre- 
sents the lowest common average meaning of the term. In actual 
use, the term means more or less than such a definition, and is 
seldom, if ever, completely satisfied by the definition. The degree 
of knowledge of any term is measured, not by recognition of a par- 
ticular shade of meaning, but by the range of variation in mean- 
ing which the individual recognizes. 

Knowledge of organized or closely related social phenomena pro- 
ceeds usually from general to detailed knowledge, and from each 
separate phenomenon to its relations with other social phenomena. 
TheF possession of advanced knowledge is measured, not by the 
recognition of a particular detail or a particular relationship, but 
by the range of details and relationships. 

Advance in social knowledge is a process of accumulation and 
integration varying greatly according to individual. Virtually 
every test tried out at different grade levels and even with groups 
homogeneous in physical or mental age, as the latter was deter- 
mined by existing measures, revealed a fairly wide range of differ- 
ences. One pupil was acquiring at one level knowledge which some 
cff his fellows had already grasped three or four years earlier, and 
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which others of his fellows were not to obtain for three or four 
more years. Since each grade level offered a different course or 
courses, the same knowledge apparently was acquired through 
different, or additional, media by the different pupils. This would 
seem to suggest that there is little basis for assigning any given 
course, topic, or term to any particular grade level. 

The accumulation of social knowledge is, in large measure, de- 
pendent upon its integration with previous knowledge possessed 
by the pupil. The retention of new knowledge is probable chiefly 
in the degree to which it is related to existing knowledge. The 
pupil who is able to relate it immediately to his other social knowl- 
edge may require no further repetition, while those pupils who do 
not relate it may require many and varied repetitions before they 
retain the new knowledge. Each pupil may be regarded as build- 
ing up a core of related knowledge. The complete integration of 
this information or knowledge to include all the elements listed in 
the second paragraph of this division is scarcely possible before the 
pupil has reached adult years. The process of integration on this 
more complete basis goes on through active life. In this sense of 
building up a core of completely integrated knowledge, school in- 
struction in social science subjects can only serve as an introduc- 
tion. The rounding out of that core, as well as its continued en- 
largement, is the work of adult years. 

The relation of social knowledge to conduct is a more complex 
process than the acquisition of knowledge alone. Much of knowl- 
edge is acquired through the medium of words, whereas conduct 
involves additional emotional and motor elements, seldom trans- 
lated into words. The indivdual’s life is a microcosm of social 
activity. His education in social conduct involves both the recog- 
nition of this fact and the relation of verbal knowledge to his own 
experience and motor knowledge. Both are gradual processes and 
require sustained instructional effort in order that the pupil may 
identify his own experiences in their proper terms and thus link 
the more remote universal knowledge to these personal experi- 
ences. Only in this way can he be brought to make the fullest use 
of his growing knowledge in shaping his social conduct. 
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The New-Type Test in the Social Sciences 

The Technique of the New-Type Test, The work of the Investi- 
gation brought the social scientist into close contact with the 
technique of the new-type test, as it related to his own subject. 
The apparent simplicity of the tests, with their short questions 
and shorter answers, had led social scientists to anticipate a very 
rapid production of tests for any of their purposes. They were 
much disappointed, therefore, by the fact that the most simple of 
the tests constructed for them required a year of time and great 
labor as well, while the very first of the tests projected, the Kelley- 
Trabue test, was not completed until the last year of the Investi- 
gation. Their disappointment was increased by the fact that the 
tests, when completed, touched so small a portion of the field of 
learning and seemed imperfect even for that area. The time and 
effort involved in the construction of the tests seemed out of pro- 
portion to the results obtained. 

The form of the tests, designed apparently to limit answers to 
a single word and questions likewise to as few words as possible, 
also proved a source of disappointment to social scientists. Such 
tests seemed to imply a verbal precision for which social science 
offered little justification. Social ideas and social phenomena are 
usually capable of description and expression in a variety of ways. 
Furthermore, such minute sampling of social science knowledge 
clearly did not constitute a test of the student’s comprehensive 
knowledge, or of his ability to develop sustained exposition of large 
ide^s and to include the conditional elements which qualify any 
but the most simple of social situations. In other words, the ex- 
tremely short answer form of the test seemed an artificial limitation 
which must confine such tests to the measurement of only the 
fragmentary beginnings of social science knowledge. 

The statistical standardization of tests brought further disap- 
pointments to social scientists. The purpose of this standardiza- 
tion was to supply in the test an instrument applicable to all 
schools of the' country at the particular grade level for which it 
had been designed. Earlier statistical analyses of the scores on the 
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trial of tests tended to eliminate those items which had proven 
very difficult or very easy. 1 his process tended to defeat any pos- 
sibility of testing the student’s range of knowledge, and to confine 
measurement to mediocrity alone. It was not uncommon when the 
tests were applied to successive grades to find the more advanced 
students doing less well on many items than those in the earlier 
grades for whom the test had been standardized. The emphasis on 
the mediocre was penalizing those students whose knowledge had 
advanced to appreciation of shades of meaning, modifying condi- 
tions, greater degree of detail, and other qualifying considerations. 
Dissatisfaction at such an outcome is expressed in several of the 
articles in this volume. 

These deficiencies in the technique of the new-type test as ap- 
plied to the measurement of social science learning so marked in 
the early stages of this investigation have received attention from 
scholars in educational measurement. Anderson and Lindquist, 
working independently, have suggested important modifications 
in the technique of standardized tests applied to history and other 
social science subjects. The discussions by Kelley in the present 
volume show that the range of the tests can be greatly extended 
to satisfy many of the objections to narrow standardization. Fur- 
ther progress may be expected through the close co-operation of 
social scientists and scholars in educational measurement. 

The New-Type Test as an Instrument of Instruction. The value 
of the new-type test as a supplementary device in school instruc- 
tion, whether during the progress of the work or in the examina- 
tions at the close of instruction, seems established. Its deficiencies, 
however, are still so many that it can not be used as a substitute 
for all other tests at either of these stages.* 

The dangers arising from an exclusive use of the new-type test 
in instruction may be briefly summarized. Teachers seldom have 
either the time or the technical skill to prepare satisfactory tests. 
Hastily constructed tests are usually limited to those precise and 
separate material elements of instruction, the study of whose in- 

ipor a more extended discussion of uses and limitations of new-type test in 
instruction, see pp. 95-113. 
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terrelationship, not the material elements themselves, constitutes 
social science. The student’s tendency to determine the aims of 
instruction narrowly from the character of questions employed in 
tests and examinations would lead him thus to miss nearly all the 
aims of instruction in this field. One of the most striking losses 
resulting from excessive use of the new-type test is the student’s 
failure to acquire practice in developing his ideas and knowledge 
in sustained, clear, and effective exposition. The other losses, 
though less striking, are just as real. 

The chief uses of the new-type test in its variant forms as a sup- 
plementary device in instruction at present may be briefly enu- 
merated. It is the most efficient device for detecting the student’s 
possession of those separate material elements which, though not 
the end of instruction, are an essential preliminary to those ends 
comparable to the shoring which the engineer employs in shaping 
buildings made of concrete. No other testing device covers so 
great a range of information in so short a time, or can be graded 
so quickly and accurately. It may also be used to discover the 
student’s knowledge of the simpler and limited relationship of this 
material. It may be used to some extent, also, in testing student's 
ability to apply ideas to new materials, and his possession of the 
skills involved in the subject. The more advanced and complex 
stages of these values, however, must as yet be discovered by 
other forms of test. 

TJif Netv-Type Test as an Instrument of Educational Research. 
The new-type test was used in this Investigation primarily as an 
instrument of educational research in the field of the social science 
subjects. Its limitations, deficiencies, and failures as such have 
been sufficiently indicated and constitute an important finding of 
the Investigation. It remains now to consider the positive results 
jf the effort. 

The joint effort of scholars in social science and in educational 
measurement led to a more careful analysis of the aims and func- 
tions of social science instruction than has hitherto been available, 
rhough the'^statement of objectives offered small comfort to those 
nIwo had hoped that it might appear in the form of particularized, 
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material knowledge, that very fact directed the attention of work- 
ers in educational measurement to the less tangible, but equally 
real, values of the social science subjects. The recognition of the 
fact that the definite and material elements in social science were 
but mean! to ends and not ends in themselves was another import- 
ant contribution of this work. 

Clear recognition of the fact that social science was primarily 
concerned with the relationship of social phenomena and the ac- 
ceptance of the implication of this fact gave a more definite direc- 
tion to such pioneer efforts as those of Barr-Gifford, Woody- 
Stephenson and the Presseys. Their lists of terms, distinctive of the 
social sciences, could be pruned of too many purely material and 
specific elements and extended toward the inclusion of all terms 
expressive of relationship. Even though such a list of single terms, 
if completed, would be inadequate for the needs of social science, 
whose more advanced relationships can scarcely be stated in single 
terms or even in brief phrases, yet the use of such lists in tests 
may be regarded as a distinct advance. 

Terms expressive of relationship are not limited to particular 
material elements or completed social phenomena, but are uni- 
versal in their application to similar elements and situations pres- 
ent or impending. Knowledge of them, therefore, may be regarded 
as of permanent value to the possessor, a definite increment in his 
accumulation of social science. The Wesley, Wesley-Gold and 
Kelty-Moore tests constructed for this Investigation, and the 
Pressey tests, authorized as a technical check upon the former, 
show that considerable advance can be made in testing knowledge 
of relationship. The several ramifications of these tests, the most 
immediate and extensive contribution of the new-type test to' 
educational research in social science subjects have been discussed 
at length earlier^ and are therefore not repeated here. 

Social scientists and scholars of educational measurement may 
not accept all the claims which the authors of these tests have 
made for them. Miss Kelty’s insistence that pupils be drilled in 
these terms at early grade levels overlooks the fact that such 

'See pp. 95-113, 
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terms have a growing and widening meaning which is cumulative 
throughout the period of formal instruction, as well as later, and 
is built up, not by one course alone, but by all courses in social 
science, as well as by life’s experiences outside the class-room. 
Her statement, however, adds needed emphasis to the fact that 
such terms are of vital importance in social science. The desired 
end may be gained even more effectively if, instead of drilling at 
one grade level, teachers at all grade levels give proper emphasis to 
the terms as they occur in their work, noting the different shade of 
meaning which the different situations involve. 

The definite list of terms set down for separate grades by Mrs. 
Pressey carries an implication which her text corrects. The terms 
listed have cumulative meaning. At most, the significance of her 
particular grade lists may be stated as indicating that the larger 
fraction of pupils in the particular grades are aware of those words. 
The possibility that pupils at the fourth grade level might be made 
aware of almost any term, and that awareness of a term may im- 
ply nothing more than a recognition of that particular combination 
of letters is more clearly indicated in her text. Her tests, like the 
others, show the cumulative enlargement of knowledge of the re- 
lationships expressed by the term through successive grades. 

The Clark tests and some of the Parker-Calkins tests, directed 
toward the measurement of skills, mark an advance in the testing 
of patterns of relationship. Whether such patterns are derived 
from the observation of recurrent complex social phenomena or 
represent logical schemes for the direction or control of social 
activity, they constitute some of the most important elements of 
social science instruction. Like single terms, few social patterns 
have sharply limited boundaries. In life itself they seldom recur 
in all details, yet they possess essentially common cores whose 
knowledge can be tested to a considerable extent. As these pat- 
terns represent more complex knowledge, the construction of 
adequate tests implies a greater demand upon the subject-matter 
specialists and suggests that the responsibility for further advance 
in the measurement of social patterns must rest primarily with the 
social scientist. 
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In the tests of attitudes and traits of personality, an effort was 
made to penetrate somewhat to those ethical and esthetic elements 
which condition nearly all social practice. Tests of attitude, long 
a preoccupation of psychologists, have been employed with some 
degree of success by social psychologists. The character of the atti- 
tude tests employed, however, and the limited extent of their 
application by this Investigation do not justify any definite con- 
clusion as to their possible modification for use in the schools. 
The Jordan study of honesty and cheating in school situations 
confirms the impression of most social scientists that these quali- 
ties are not definite traits of personality, but, rather, reactions to 
immediate situations. 

The general recognition of individual differences has served as a 
strong incentive to discover, if possible, the inherent qualities and 
traits of personality of which such individual differences are com- 
pounded. Though the social scientist is primarily interested in the 
social expression of personality, he would welcome the psycholo- 
gist’s discovery of any inherent and presumably fixed traits of 
personality as the basis of such social expression. It was with this 
hope that the Kelley-Trabue study was undertaken. Whether the 
traits selected are truly permanent traits of personality, and 
whether the results obtained are of value to social science, only 
further studies can demonstrate. The authors of the test are very 
modest in their claims for the results obtained. Their use of a free- 
association technique which permitted the subjects of the test the 
widest possible freedom of response must commend itself for its 
intellectual courage. Such procedure obviates many of the objec- 
tions to the usual form of the new-type tests which limit the re- 
sponse of the subject within very narrow boundaries. 

The use of the new-type test as an instrument of educational 
research in this Investigation has indicated that it can be made to 
yield additional data of value in conditioning educational policy 
and procedure. That data thus accumulated are not any more 
determinative of policy and procedure than data accumulated by 
other methods of research Beard has clearly indicated. The use of 
the new-type tests in social science subjects must be considered in 
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competition with other methods of research, and the prospects of 
returns from its use must be weighed against the great cost and 
effort involved. 

TerUative Character of Present Conclusions. This volume is pri- 
marily a record of experience in the use of the new-type test in 
connection with social science instruction. The studies which it 
contains reveal the efforts of teachers and scholars in various sub- 
ject fields of social science and of scholars in educational measure- 
ment. They reflect the background from which each of the authors 
has approached his task. So much progress has been made in the 
technique of testing, as well as in other fields of scholarship, since 
this work was begun that it is possible that many of those who have 
contributed to this volume would proceed quite differently were 
they to begin their work now. Those who have worked at the ap- 
plication of the new-type test to the problems of instruction in 
social science, either in this Investigation or pari passu with it, may 
be surprised that some of the ideas recorded in this volume were 
ever held by themselves. Yet the fact that they were so held 
suggests that others less experienced may also entertain them. A 
faithful record of our experience with tests in the social science 
field, however, revealing the story of hypotheses entertained and 
abandoned, conclusions reached and discarded, seemed in itself 
a contribution of value. If this record of our efforts enables future 
workers in this field not only to avoid the false steps here recorded, 
but also to utilize any advances indicated, it will be doubly useful. 
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A DIVERGENT OPINION AS TO THE FUNCTION OF 
TESTS AND TESTING 

T. L. Kelley 

The use of tests in connection with the social studies has been 
widely discussed and has given rise to sharp differences of opinion. 
It seems desirable in this appendix that the background and re- 
sulting views of those who in general object to the use of tests 
in the social studies be stated in contradistinction to the back- 
ground and views of those who in general advocate the use of 
objective tests in the social studies. 

The writer has found it convenient to refer to the former as 
opponents and to the latter as proponents, and to use the letters 
“o” and “p” to designate these groups. The writer has ample 
reason to believe that the opponents as here characterized repre- 
sent a very real and substantial group. He would classify himself 
as a proponent and has made bold to use the editorial “we" in 
stating the views of proponents for he has much reason to believe 
that the views so presented are widely held by experimentalists 
in the fields of psychology and the social studies. 

It is hoped that antitheses drawn are so opposite that the views 
of opponents and proponents are fairly bracketed, but it is hardly 
to be expected that a comprehensive picture of any particular 
person’s view will be entirely within the bracket thus formed. In 
fact, due to space limitations and attempts to simplify presen- 
tation, there are some very pertinent aspects of the view of the 
writer which are not touched upon {e.g., the meaning and measure- 
ment of “reliability” as applied to trait and achievement meas- 
ures). Though this is a report by a “proponent,” it is not presented 
as an extreme view, but as the deliberate conviction of the write”. 

We will then in short statements labelled (p) and (o) enumerate 
elements in the frame of reference of (p) the proponents, and in 
that of (o) the opponents. 
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(l-o) The opponents assert a close intimacy between teaching 
and testing regardless of the purpose of instruction. 

(1-p) With this we agree but would emphasize these related 
uses of tests: (a) their use in the immediate facilitation of instruc- 
tion of the pupils taking the test, (b) their use for the advance- 
ment of professional knowledge which will serve in the solution 
of various instructional problems, and (c) their use as instruments 
in any continuing, or fairly long time, guidance program. We 
would also emphasize that although every teaching or learning 
process implies a program of testing, it requires a separate focus 
of attention and frequently an additional and definite act to realize 
this program. To assume that the testing function is realized as 
soon as the teaching purpose is defined and content presented is 
education by faith. We specifically deprecate an educational pro- 
gram which does not involve some definite and adequate appraisal 
of pupil growth and accomplishment. 

(2-o) The opponents manifest a general aversion to intelligence 
tests and all things for which they have been used, such as homo- 
geneous grouping. 

(2-p) In a number of fields including certain ones connected 
with the social studies, adequate knowledge as to the function of 
a general intelligence measure is as yet lacking, but we commend 
the use of the measure together with more specialized measures, 
for the purpose of ascertaining their limits of utility. 

We admit to something of an aversion to intelligence tests, for 
the specific reason that they have been and are widely used in 
instances where purposes would be demonstrably better served by 
achievement tests of one sort or another. The intelligence measure 
is an omnibus affair that washes out detail. If the detail can be 
given with fair reliability by teachers’ judgments or discriminat- 
ing tests, we believe that it is always advantageous that it be 
given. We do not beliei^e, (f.g., that a homogeneous grouping or 
other adaptation to ifidividual needs on the basis of any omnibus 
measure will ordinary classroom needs as well as a grouping 
or an adaptation which is more discriminatory of special abilities, 
^e^neral intelligence test measure is the traditional measure, 
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and should be replaced in the social studies by new measures 
adapted to the particular problems of group guidance and instruc- 
tion of the social studies. It should not, however, be replaced by 
judgments or new test measures greatly inferior to it in reliability. 

(3-o) The opponents state that testers rate certain vocations 
as more important or valuable than others. 

(3-p) This shows a misunderstanding of the significance of 
findings from various tests that there are substantial mean differ- 
ences in accomplishment of members of various vocations. These 
mean differences do not determine the position of “an” engineer 
or of “a” skilled artisan. Neither do they imply anything as to 
the relative “importance” or “value” of the various vocations. 

(4-o) The opponents show no awareness of the tests of relia- 
bility and validity of measuring instruments, either judgments 
of teachers or of test scores. 

(4-p) We believe that such awareness is essential to any edu- 
cator who is not content to work in the dark. 

(5-o) The opponents show but little awareness of the distinc- 
tion in function and limitation of different sorts of appraisals 
and tests, such as intelligence, achievement, character trait and 
attitude tests, etc. 

(5-p) That teachers’ appraisals of the achievement and capac- 
ities of pupils are characteristically woefully unreliable is ac- 
cepted as a sad fact which cannot be remedied by any simple 
means and to which educators must adjust if a serious effort is 
made to adapt instruction and counselling to individual needs. 
That some of the so-called most important characteristics of people, 
namely their character traits, are still less reliably appraised either 
by judgments or by available tests, is a second sad fact to which 
one must adjust. It is futile to speak glibly of an education in- 
volving traits whose very existence in different amounts from 
pupil to pupil remains unknown. If certain trait appraisals have 
some, but only a little, credibility, it still remains relatively futile 
to build an elaborate educational structure upon them. The full 
extent and depth of our ignorance of character traits and their 
functioning is quite beyond the credibility of professors of the 

[ 489 ] 



TESTS AMD MEASUREMENTS 


social studies, and even of elementary and high school teachers 
who have not attempted to measure traits. Any structure built 
in ignorance of this is built upon quicksand, and accordingly ef- 
forts directed toward the better measurement of these traits are 
of prime importance, although such efforts be but partially suc- 
cessful. 

(6-o) The opponents assert that objective tests can only meas- 
ure the mechanical aspects of instruction and achievement. 

(6-p) With this we unqualifiedly disagree. Any aspect of in- 
struction subject to appraisal by teachers (and we affirm that all 
other aspects are pot shots in the dark and unworthy of being a 
part of recommended practice, but only of being objects of in- 
vestigation) is ordinarily more accurately appraised by the con- 
sensus of several teachers or associates than by one; is more ac- 
curately appraised when judgments of these several are carefully 
combined into an average than when lumped together by “gen- 
eral impression;” and frequently is demonstrably more accurately 
appraised by tests of one sort or another than by the judgments 
of teachers and associates, even though two or three such judg- 
ments may be involved. These statements hold “on the average,” 
for of course occasionally some single person’s hunch will strike 
closer to the truth than any corresponding objective measure- 
ment, however refined, for complete universal accuracy is never 
to be expected. 

In general the progressive lessening of the reliability of tests 
.as the field of measurement moves from the mechanical to the 
spiritual aspects of a subject is granted, but it is probably not as 
great as the corresponding decrease in the reliability of teachers’ 
judgments. This is a matter that can be experimentally deter- 
mined in any specific case, provided that the trait in question is 
capable of being sensed by more than one person. The teacher 
claiming a unique and divine insight as to some characteristic of 
a pupil is clearly not subjecting himself to test, and equally clearly 
is making^ a claim that cannot be accepted by another as valid 
except it be taken on “authority.” The first and greatest need in 
connection with modifying pupils with respect to some “higher” 
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aspect is knowledge of the certainty with which various tech- 
niques, — teachers’ judgments, tests of various sorts, etc., — indicate 
the existence of, or the amount of, these higher traits in pupils. 

(7-o) The opponents assert that the use of objective tests di- 
verts the attention from those aspects of education which are most 
important. 

(7-p) With this we agree in the sense that test makers have 
frequently not pointed out the limited area covered by their tests, 
and that certain narrow-minded, easily satisfied teachers now 
exist, and may continue to exist under a new social order, who 
are prone to be content if the mechanical aspects of their subjects 
are mastered and if they have the supporting evidence given by 
tests that this is so. Such teachers will not be made efficient by 
withholding such instruments of precision as are available and as 
cover a part of their field of instruction. Such withholding would 
merely extend their field of inefficiency to include the more me- 
chanical as well as the more subtle aspects of instruction. So far 
as it is possible the remedy for their narrow-mindedness lies in 
the criticisms of an intelligent supervisory staff. 

For teachers with a vision beyond mechanics, trustworthy ob- 
jective tests of the mechanical aspects will prove a boon. Were 
members of a class variable in such things as strength requisite 
to holding a piece of crayon, hearing sufficient to follow an ordi- 
nary conversation, eyesight required to read the common printed 
matter of the classroom, etc., the teacher simply asking a pupil to 
draw a map on the blackboard would find himself balked because 
pupils would fail, due to elementary or foundational conditioning 
factors. For the teacher to be unable to count upon strength to 
manipulate a piece of chalk, etc., would hinder the progress of any 
ordinary assignment in geography, and it would make impossible 
an assignment calling for undivided attention and a careful train 
of consecutive thinking. Only after all of these elementary condi- 
tioning things have been accounted for by homogeneous group- 
ing of pupils, — by segregating or special handling of those lacking 
the requisite sufficient ability, — ^is it possible to devote undis- 
turbed and concentrated thought to the intellectual problem which 
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is the matter of Brst importance. The expert teacher having ade- 
quate tests of vocabulary, or reading ability, etc., and having 
determined the mean status of the pupils of his grade, and the 
important individual differences existing with reference to knowl- 
I edge of concepts, reading skill, and other mechanical factors, can 
now devote to the other issues of his subject an attention which is 
pointed and fruitful because conditioned by a knowledge of the 
foundation upon which to build. The objective measures have freed 
him from the bondage of devotion to the content they represent. 
Thus for the better teachers and the better methods of instruction 
the writers categorically disagree with the tenet that objective 
measures, even of mechanical phases, divert the attention from the 
more spiritual ends capable of realization through the subject. 

The implication is here present that these tests of mechanical 
phases are not artifacts but measures of understandings and abili- 
ties which are needed in the existing world and will probably be 
needed in the world to which the pupil is being introduced. No 
defense is made of tests of which this is not true, and the determi- 
nation of the fitness of the subject matter of the test is the joint 
responsibility of masters (1) of the subject in question, and (2) 
of the psychology of the levels of intelligence to which it is in- 
tended to present or teach the subject matter in question. While 
specifically affirming the necessity of subject matter specialists in 
connection with the grade selection made of test items, we equally 
affirm the necessity of first hand knowledge of the past achieve- 
ment and present powers of children at the levels which it is pro- 
posed to teach and to test. We do not believe that the duties of 
the citizens-to-be are so simple in their demands upon understand- 
ing, so congruent with unmodified native tendencies, and so easy 
of execution that no problem of education exists except that of 
indoctrination and the measurement of loyalties. We look at spe- 
cific indoctrination and specific loyalties as incidents in the flow 
of human life that will be found to be running its course long after 
these tl^^fecific doctrines and attachments have been replaced by 
thojif'inf a still newer and more enlightened age. Meanwhile in 
the process of getting from the past to the future, we believe that 
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measurement of the loyalties considered highest at each step is 
an appropriate part of any tuition concerned with them. We would 
go further and advocate the measurement of collateral and an- 
tagonistic loyalties in order to enable an intelligent direction of 
whatever type of educative process is undertaken. We believe 
an attempt should be made at a definite evaluation of these loyal- 
ties, either by means of the judgments of competent people or by 
test measures which are demonstrably pertinent and of superior 
reliability. 

The clause “demonstrably pertinent and of superior reliability” 
is the crux of this statement, for no test measure is axiomatically 
entitled to confidence, but only after its pertinence and reliability 
have been established or made highly probable by careful study. 
A granting of the entire argument of this report should not auto- 
matically lead to the belief in the efficacy of any single objective 
measure, — each must establish its field of usefulness. 

Such belief should not rest upon the title of the test nor the 
name of its author but upon measurable agreement of test results 
with the judgments of teachers and other intelligent people, — 
the more the better, — acquainted with the persons tested. Though 
the test has a reliability indicated by a coefficient of .90, and 
teachers’ judgments (for this trait in hand) of but .40, nevertheless 
the latter provide the standard whereby the validity of the test 
for the purpose in question is to be judged. Well established pro- 
cedures for doing this are available. 

Given, in an experimental situation, three measures for each 
pupil of a group: {a) a criterion measure of a trait X based upon 
the consensus of opinion of several competent people, (^) a single 
teacher's estimate of X, and (c) the score on an objective test 
purporting to measure X, and given the experimental finding that 
one of the last two measures tends to agree more closely with (a) 
than does the other. Then in a new situation wherein {b) and (c) 
only are available, it is the claim of test advocates that the measure 
which agreed the more closely with (a) is the more trustworthy 
measure of X. The ground for belief is thus simple and straight- 
forward and has no mystical elements in it, and no prejudice in 
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it favoring the test measure as against the teacher’s judgment 
measure. 

(8-o) The opponents assert that individual differences are un- 
important in connection with the broad purpose of a group con- 
cerned with fundamental curriculum revision in the social studies, 
which purpose should be primarily to create or prepare for a better 
social order. 

(8-p) We unqualifiedly disagree. Directing the attention to 
individual differences in mathematical ability, musical ability, 
memory, self-control, affection, sociability, etc., there is evidence 
of disparities from individual to individual which would make 
such differences as those in height seem altogether trivial. If 
differences in mentality (either in general intelligence or specific 
abilities such as musical capacity, mathematical ability, memory, 
etc.) were reflected as differences in height, the world would be 
peopled with two foot dwarfs and twelve foot giants, with a heavy 
representation of all grades in between. This is the range of in- 
tellectual equipment with which we as educators are concerned, 
and to think of it as unimportant in connection with the so-called 
broad purposes of any group concerned with the fundamental 
problems of education is merely to characterize these purposes as 
pertinent to some other world than the one in which we live, or, 
with considerable plausibility, than the one in which our children 
and grandchildren will live. 

The opponents conceive the purpose of such a group to be 
jto create or prepare for a better social order. However, it is quite 
thinkable that a greater purpose would be to provide instruments 
for knowing the social structure, methods of appraisals of public 
servants, channels for the free flow of opinion and of information, 
and above all methods for experimentally verifying social hy- 
potheses. Were this the purpose it would obviously change the 
entire effort of the undertaking which would then be to nurture 
rectitude of procedure and of thought in innumerable self-germi- 
nating centers of social reconstruction, rather than to be the initial 
germ center and the structural grandparent of all such endeavors. 

It is claimed by the writer that a group committed to broad 
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purposes which do not include a study of individual differences is 
not qualified to pass judgment upon objective methods of meas- 
urement, for a correct appraisal of tests is to be expected only after 
a serious attempt to use them for their avowed ends has been 
made. The real criticism of a test is not to be made in terms of- 
what the items “look like" on paper (suggestive though this may 
be) but in terms of the light the use of the test throws upon pupil 
development or behavior. 

Objection to an approach dominated by the idea of experi- 
mental verification of conclusions is within the province of any 
deliberative body, which, however, must expect any of its own 
conclusions to be untrustworthy until they have been experi- 
mentally found tenable. The unremitting research and patience 
in awaiting outcomes of the scientist will alone serve to add the 
element of trustworthiness so necessary to pragmatic truth.* The 
words of John Dewey* well express the necessity for a fruiting in 
experience: 

“In holding that the values which should determine the direc- 
tion of education can be dug out of life-experience itself, we are 
denying by implication the position taken by some opposed types 
of philosophical theory. We affirm that genuine values and tenable 
ends and ideals are to be derived from what is found within the 
movement of experience. Hence we deny the views which assert 
that philosophy can derive them out of itself by excogitation, or 
that they can be derived from authority, human or supernatural, 
or from any transcendent source.” 

(9-o) The opponents assert that the forthcoming social order 
will be different from the present in important things of the spirit 
and that these cannot be measured by objective tests. Any reliance 
upon them is at best a concern with the trivialities of the nefr age. 

‘This point is treated of in considerable detail in Chapter 7, The Conference 
Method of Finding the Truth, of Scientific Method, T. L. Kelley, 1932, MacmUlan. 

•Chapter 9, p. 294, The Educational Frontier, Ttoenty-firsi Yearbook of the 
National Society of College Teachers of Education, 1933. Thif yearboolt, representing 
the conference product of a group of educational philosopher*, it delightfully 
naive as coming from those whose slogan is integration, in its inconsistency from 
chapter to chapter. The present writer admits ability to quote from this same re- 
port statements of just the opposite vein from that of the above sentences of John 
Dewey. 
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(9-p) Granting a probable difference in social outcome as great 
even as a reversal of attitude toward private property, never- 
theless, we do not grant a probable basic difference in the instincts, 
satisfactions, intelligences, skills, and types of social contact which 
will be ordinarily manifest in the new order. Further, we do not 
grant the impossibility of measuring by semi-objective means the 
new traits or accomplishments which will be deemed socially de- 
sirable. We do grant no present instruments for measuring these 
new things and very inadequate instruments for measuring most 
of the traits and accomplishments essential to happy living to- 
gether in our present social order. 

(lO-o) As an estimate we would say that the opponents assert 
that the new elements in the mental life of the new social order 
bear a ratio to those in the present social order of about 9 to 1 
in importance, if not in number and magnitude. By inference, no 
tests, however well adjusted at any given time, could ever be 
acceptable for a future time. 

(10-p) We believe that tests can be changed in content and 
emphasis much more rapidly than the mental content of the mass 
of school or adult population can be changed, in this or in the 
new social order. 

In the case of a growing thing, say a tree, the sap wood may 
bear a ratio to the heart wood of much less than one to nine, 
but still be of greater relative importance than nine to one in 
determining the nature of future growth. It, in short, is the only 
4 )art that is growing. Does a parallel hold with reference to mental 
development ? Presumably so with reference to changes in the 
views of adults wherein tradition, the heart wood, is massive. 
For example, thought and action in harmony with the capitalistic 
system may be second nature to the informed adult and not readily, 
or in the main, amenable to modification. To change such thought 
and action a tiny toe-hold via, say, a local cooperative store patron- 
ized but a few minutes a day may be more potent as determining 
growth than the mass of accepted ideas that daily control most 
of conduct. Here the new element in the individual’s mental life, 
though bearing but a small \ratio to the total, may bear a large 
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ratio to it as a determiner of growth. Should we apply the same 
view to school children They are sprouts with negligible heart 
wood. They are undergoing a process of growing into the existent 
economic system in all its phases durable and transient. At the 
same time they are growing into innumerable other systems many ' 
of which, such as the spoken and written language of the coun- 
try, tolerated conduct with reference to contacts with the stronger, 
the weaker, the brighter, the duller, the older, the younger, etc., 
call for an immeasurably greater thought and anguish than does 
the establishment of an attitude toward a church, a party, or a 
remote authority of any sort. The heavy assignments upon youth 
may be expected to continue into the new social order and the 
heavy duties of education may be expected to be the same in it 
as in the old order. The adult forgets the virginity of youth and 
that that which to him is prosaic and established is a vital phe- 
nomenon of life to the child. We therefore believe that to the 
infant, the child, the youth, and even to the late adolescent strug- 
gling to equip himself to render service that society will reward, 
there will be demands upon his mentality and his other develop- 
ing powers which will be very similar to those at present. We do 
not subscribe to the idea that the demands in question are triviali- 
ties now nor in the generation to come. 

(ll-o) The opponents recognize no real need for proof of the 
attainability of any purpose as a prerequisite to the belief in and 
acceptance of that purpose. 

(11-p) We assert that the first need of any objective concerned 
with the modification of school children is evidence as to the at- 
tainability of, or as to the conditions mediating attainability of, 
the changes m children postulated as congruent with the obj^tive. 
We admit purposes lacking proof of attainability (more accurately 
tentative hypotheses) as appropriate to trivial issues, and those so 
urgent that time for study is not present, and as appropriate to 
investigational programs of all sorts, but not as appropriate to a 
program defended as well considered, desirable, and deeply affect- 
ing many people. 

We grant that the attainability of remote purposes cannot be es- 
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tablished until the check-up at the remote time is made, but we 
affirm that until such check-up is made, the very reality of the 
purpose is questionable. A purpose that is uncertain is a purpose 
only in a dialectic sense, not a purpose in the world of sentient 
beings and present realities. That it is or is not attainable is un- 
known until the check-up has been made. The check-up is the other 
part of any serious undertaking, and a program that neglects it is 
not even half a program, A complete program involves purpose, 
check-up, and repercussion, of which the last is likely to be the 
most constructive element. We admit an aversion to remote and 
recondite purposes, and would at all times endeavor to phrase 
problems so as to be concerned with contiguous purposes and se- 
quential progress from present realities and known resources to 
clearly stated mediate objectives. This is neither opportunism on 
the one hand nor fanaticism on the other. 

(12-o) The experimental method is inapplicable as a means of 
investigating the broad purposes conceived to be important by 
serious educators, and the minor purposes are relatively trivial 
and not worth the thought of such educators in the present time 
of social stress. 

(12-p) Such comprehensive purposes as consist of an effort to 
secure non-specific outcomes cannot be investigated by any means, 
— experimental, logical, or historical. It is only as they become de- 
fined in terms of specific ends or broken into aspects with definable 
consequences, that methods of investigation appear and become 
JBvorthy of pursuit and of trust. Definable consequences are not 
verbalisms such as “to be a good citizen,” “to seek the social 
good,” etc., but rather acts of performance which can be observed, 
such “going to the polls to vote,” “serving as a boy scout mas- 
ter,” “enumerating the constitutional duties of the governor of the 
state,” “participating in the sessions of the better government 
lej^ue,” etc. The labor involved is the only deterrent to investigat- 
ing any purpose broad or narrow having specific outcomes in be- 
havior, experimentally by means of carefully controlled conditions 
of sampling, of teaching, and of measurement. Usually, a synonym 
for broad purpose is indefinite purpose, and as long as the indefi- 
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niteness remains, we admit that the ex{>erimental method is inap- 
plicable. 

(13-o) The opponents ignore the uses and results of tests and 
other forms of appraisal in educational guidance and in vocational 
counselling. 

(13-p) We consider educational and vocational counselling as a 
single function progressively becoming more important as scho- 
lastic differentiation takes place, and as higher and higher grade 
levels are reached. It is, we judge, the most intimate service to be 
rendered by the social studies. The inductions of the child into the 
political order as a citizen and into the economic order as a con- 
sumer have elements of similarity from pupil ro pupil which are 
greatly in excess of those connected with his induction into the so- 
cial order as a producer. The elements of similarity in the first twe 
cases simplify instructional problems, but the importance to the 
child, and indirectly to society, of the third induction is so great 
that we consider an educational scheme that neglects this issue at 
derelict. As it enters in, it quite naturally becomes a part of th« 
social studies program. To serve its needs an individual contact ol 
rather a new order is called for between the teacher or counseloi 
and pupil. We believe that this contact is of first importance and 
that objective tests, in so far as they are discriminatory and relia- 
ble, are of peculiar merit in its establishment upon a sound basis 
Many tests that serve this purpose will also serve other educationa 
purposes, but certain tests such as interest tests have a superioi 
value here to that in connection with other educational practices 

(14-o) The opponents assert that “educators,’’ psychologists 
and statisticians are trying to “run things” by means of tests o 
their own devising. 

(14~p) One parent of this distrust may be incompstence o 
“educators,” etc., but surely its other forebear is i gnoran ce 
Academic men in the field of the social studies in universities d< 
distrust measuring instruments, the derivation of which they d< 
not understand, and the reliability of which they cannot person 
ally determine. However, as the only reason for this lack of abilit; 
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upon their part is their indisposition to study the techniques in- 
volved, their view is hardly entitled to great respect. 

The willingness, the avidity, yes, the rather pitiable confidence, 
with which school men accept objectives given them by scholastic 
authorities, should absolve them from any desire to run things, 
except of course the machinery of the educational plant which they 
could not avoid running if they would. It is unfortunately true that 
“educators” poorly qualified in subject matter have rushed into 
print with subject matter tests. It is a matter of personal knowl- 
edge to the writer that in many instances where this has happened, 
it has followed a preceding serious but unsuccessful attempt upon 
the part of the educator to get the co-operation of academic au- 
thorities upon subject matter. The educator is frequently put in 
the position of being called upon to do something, and if univer- 
sity recluses giving courses upon the essentials of X, and how to 
teach X, refuse to concern themselves, either individually or in 
collaboration with public school men, with the measurement of X 
at elementary and high school grade levels, they should expect the 
educator to do something not to their liking. 

The statistician is concerned with the construction of objective 
test only as with a minor chore. 

The psychologist is brought into the picture, and rightly so, as 
an expert upon child capacities, just as is the subject matter special- 
ist as an expert upon subject matter. The very nature of the prob- 
lem calls for co-operative endeavor. 

' . Generally speaking, there is a real difference between the atti- 
tudes of educator and social scientist. The former desires to co- 
operate, the latter to teach the value of a co-operative social order. 
However, we disapprove of any attempt to construct measuring 
instruments for different scholastic fields which do not involve the 
careful and critical assistance of competent specialists in these 
fields. 

(15-o) The opponents fail to recognize that a multiplicity of 
5emi-c(^(cting purposes and objectives should be encouraged; 

fail to recognize that a corresponding number of variously 
ilpi^ioned tests is desirahH 
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(15-p) If the object is to help investigators ascertain conse- 
quences and purposes congruent therewith, or purposes and conse- 
quences congruent therewith, for it matters not which is conceived 
to be the starting point, it is apparent that many purposes will be 
postulated and many practices, including those actually or seem-' 
ingly quite inharmonious, will be studied, and the significance and 
relationships which are imposed by the very structure pf our 
psychological beings and by our human culture determined. The 
opponents think of efforts at social betterment and educational 
reconstruction as of the nature of an acid biting its way through 
to a pre-determined end, and accordingly one but half believed in, 
and fail to appreciate that efforts would probably serve a greater 
purpose if they were of the nature of a catalyzer, accelerating so- 
cial growth and enabling reagents to combine into inherently 
harmonious patterns. 

The writer has long maintained that for each of the disciplines 
represented in the social studies, different purposes, though each 
sponsored by reputable scholastic authorities, should be accepted 
as a basis for constructing objective tests, and that having these 
the status of school children be measured and the changes in status 
as correlated with changes in instructional techniques be experi- 
mentally determined, and that then and only then would one be in 
a position to formulate defensible and promising neo-educational 
doctrines of purpose, practice, and outcome. Present and future 
parties to curriculum reconstruction should feel a need for abun- 
dant and varied knowledge of individual and social facts and en- 
deavor to supply this knowledge expeiimentally. 


[5011 



APPENDIX 11 

TERMS IN THE SOCIAL SCIENCES 

Tliit list ii a tampUng of terms exprecsing relationship. It is, therefore, neither 
a complete list of terms used in social science nor even of terms expressing rela- 
tionship. Nor does it contain all the important terms expressing relationship. 
See Chap. 1. 


Wesley Tests, 
Editions No.* 


3 ■. >. s 

o. .A Jtl! 

1— I c a « £ 


abbey 

1 

-.227 





X 

abbot 

1 

-.029 






abdicate 

i.s 

+ .463 

X 


X 


V zz 

abhor 

1 

+ .168 




• • 


ability 

s 





e * 


abolish 

1 

+ .884 




s s 

. y 

abolition 

1 

+ .648 





. zz 

aborigines 

1 

-.438 





X 

abridgment 






X 


abrogate 

i 

-.774 

X 

X 




absentee 

1 

+ .367 






absentee landlord 

1 

+ .367 




. X 


absentee ownership 

I 

-.503 






absolute ruler 

.... 




. 

X 


absolution 

2k 


X 





absolutism 

1 

-.714 





X 

absorption 

1 

+ .215 

. 

. . 

. . 

X 

. 


*Thc lilt lymboli in thii column indicate those listi in which a given terra appeari. Theie aymboli 
foilomng meaningi; 1, The Firit American Historical Aiiociation Liit of Social Studies 
S, Mill Ware’i Word Liit of Government and Law Terms; 2e, Miss Ware’s Word List of 
2i, Miss Ware's Word Lilt of Sociology Terras; £, Eubank's “The Concepts of 


*This rolumn shows the weightings given the terras in the First American Historical Association 
List of Social Terms. 

^An “a" in this column indicates that the word also appeared in Dr. Wesley's List of Political 
Terms. 

*“xs’’ in the five columns under this heading indicate that the term was used in those editions 
of Dr. Wesley’s Test. 

*An ’’x’’ in this column indicates words which occurred in the first version of the Wesley College 
Test io the Social Studies but were not included in the published test. 

*‘’x“ in this column indicates those terras used in the published form of the Wesley College Test 
in the Social Studies. 

^An “x" in this column means that the term occurred in the original tCelty lists. A number 
shows that the term occurred in the original Kelty tests, the number indicating the grade level in 
which used. A "y” indicates that the term was used in the Kelty-Moote try-out forms. A %’’ thowi 
that the term'b used in the Kelty-Moore final forms. 

*"x** in this column marks words occurring in Dr. Preisey’s original list of 1,444 technical wotdi 
uacd in the tociel studies. This list was constructed from the following eourcei: 

i l) An analysis of 23 texthooki ii# history and civics for use in grades 4-16. 

2) An a^lytit of 6 hisiory ttitt isidely used at the high school level, fot which a frequency 
count was kept. I 
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Word 


1 

B 



abstract 

1 

- 222 

academic freedom 

1 

-1 213 

academy 

accede 

1 

-.417 

acceptance 

1, 2c 

+ 479 

accessible 

1 

+ .536 

acclaim 

1 

+ .125 

accolade 

1 

-1 633 

accommodation 

2s. E 


accommodation groups 

2s, E 

, 

accountable 

. 

account, checking 2e 

account, special interest 2e 


accounting 

1, 2e 

+ jio 

accredited 


< • . • 

acculturation 

2s,' E 


accumulated reserves 
accumulation 

1 

+ .573 

accusation 

l.S 

+ 70S 

achievement 

1, 25, E 

+ .742 

achievement, collective 

2s 


acquiesce 

1 

- 221 

acquired characteristic 

1 

-1 263 

acquisitiveness 

acquisition 

1 

+ 120 


acquit X 


Wesley 

Wesley Tests, College 
Editions No. . — * — . 



JS 

s 

D. 

C 



X 

X 


X 


X 


X 



'The Vocabulary 


Ykf JovrntU of Editcational JUuarej^ XX. 103-121^ 1929|^and 
he Special Vocabulary of Civici," Thi Journal oj Educational 


(3) Similar atudiei in hutory and civici A. S. Barr and_C. W GilTorii 
of American History," Ti 
A. W. Stephenson, “The 

XesearcA, XVIll' 297-304. 1928. . 

(4) Frequency counts of special words occurring on the front pages of newspapars, in the 
editorials, and in many articles in magaunes dealing with .current events. _ 

"y” marks words included in the list of 4IS constructed from the original list of 1444, uaing the 
following criteria. 

(1) Frequency of appearance. .... . > 

(2) Combination of ratings made by 64 teachers of secondary school hist^, and 5 pro- 
feasts of coll^ history, as to whether the words were essential, "accessory, or 

(3) R8tln*g?^by"* especially trained individuals (a college professor whose specialty it pro~ 
feasionaliz^ subject-matter in hutory, one college profMsor of histonr, an espenmeed 
and intelligent social worker, a mature high school principal who had taught fatstory 
for several years, a public school specialist in educational 'tteaich, a tp^liat m ^ 
conitmction* cKc lutVior of tH® tttw} n to lociolojicii vsloc of c^cn worog ” 

is, its use outside the history classroom. l i- e u i *i i 

Any wordJ not meeting all three criteria were eliminawl from this list of absolute ^tUs. 
"rs" indicates words included in try-out Forms A, B. C, D, but not m final forms A and B. 
"Z" desigiiaeei words included in final Forms A and B of the test, with the following qualificatsons: 
*‘zX” marks words used as sample 
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Word 

J 

e 

s 

•e 

Z 

*A 


Wesley Tests, 
Editions No. 

Wesley 

College 



w 

v» 

§ 

«-* 

tm 

o 

s 

V 

1-^ 


^ E > 

N-4 W M 

Unpub. 

Pub. 

Kelty 

Prcssey 

act 

2g, S 








z 

action (activity) 
corporate 

E, 2s 






X 



actionable 







X 


, 

activities 

2s, S 


X 



■ . • 




adaptation 

2s, E 





X X 




adherent 

1 

-.206 

X 

X 

X 

X X 



X 

adjourn 

1. s 

+ .S84 

X 

X 





z 

adjudge 

1 

-.613 

X 







adjust 

1 

+ .584 

. 






, 

adjustment 

2s, E 






X 



administer 

1. s 

+ .17n 

X 






, 

administration 

1. S, 2g 

+ .347 

X 






z 

administrative 

department 

(2g) 









administrative functions 

• . • • • 





X 



administrative law 

2g 









admiral 

1 

+ .490 

X 






X 

admiralty 

2g 








X 

admission 









X 

admission of new state 

2g 








, 

adolescence 

1 

-.391 







, 

adoption 

1 

+ .446 

X 






X 

adult 

S 








, 

adultery 

1 

-.223 







, 

ad valorem 

1 

-.913 





. X 



advance 









X 

advancement 



X 

X 





• 

advertiser 

2c 









advice and consent 


. 





X 



advisor 









X 

advocate 

i 

+ .047 

X 







aerial 









X 

aeronautics 









X 

affairs 

s 








X 

affection 

1 

+ .547 






6 


affecdonal 







X 



affidavit 









X 

affiliation 

i 

-.069 

X 







affirm 

1 

+ .400 








African v 







X 



aftermath 

. . • . 








X 

age 

.... 








X 

agency 

i,s 

* f .5840 







X 
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Word 

List Symbol 

Morton Weights 

^ Wesley Tests, 

.» Editions No. 

Wesley 

College 

a . 
§* ■§ 

b 

11 

& 

^ - S 


III 

IV 

agenda 








X 

aggrandizement 

1 

- 801 

X 

, 


, , 


• 

aggregate incone 






X 


• 

^KSregate mass 



. 



X 



aggregation 

2s, E 







z 

AKSression 



. 





X 

aggressor 



. 



X 


. 

agitation 

1 

-f 531 

X 





X 

agitator 

1 

+ 405 

X 





• 

agrarian 

1 

- 922 

X 





X 

agreement 



X X 





• 

agricultural labor 

1 

+ 432 







agricultural revolution 

1 

- 619 







agnculture 

1 , s 

+ 722 






z 

aide-de-camp 



• • ♦ 





X 

airplane 

1 

+ 289 






X 

alderman 

1, 2g, S 

-E 010 

X 



X 


X 

alien 

L 2b, S 

+ 416 

X 



X 


z 

alienate 

1 

- 279 

. 






alignment 








X 

allegiance 

1, Zg, S 

+ 169 

X 





X 

allegation 








X 

alleviate 

1 

- 454 



X 




alliance 

1, 2s, S 


X 





z 

alliance regime 


, 

• • » 



X 



allies 

S 







z 

allot 

1 

- 269 

■ ■ • 






allotment 


• . • 




X 


■ 

almoner 


. 




X 


f 

almseiving 

LS 

-f 216 




, , 


• 

almsnousc 

l.s 

+ 215 

X 


• « 

, , 


X 

altar 

1 

+ 663 




, , 



alteration 

1 

-E.3S3 




, , 



altering 


• • * 



» * 

X 



altitude 





• r 

, , 


X 

altruism 

1, 2s 

-.556 



X X 




amalgamation 

2s. E 





X 



ambassador 

1. 2g, S 

-i- 095 

X 



X 


z 

ambush 








X 

amendment 

1. 2g, S 

+ 332 

X 



, , 

X 

z 

America 

S 







* 

American Confederation 








of Labor 

2e 








Americanization 

1 

+ .174 

X 



X 


• 
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Word 

List Symbol 

Morton Weights 

Wesley List 


Wesley Tests, 
Editions No. 

Wesley 

College 

3 

A ^ 
^ a. 

Kelty 

Pressey 


11 

\iy4 

in 

IV 

amicable 








X 

amity 

1 

-.887 






. 

ammunition 

1 

-b.853 

X 





X 

amnesty 

i.s 

-.946 

X 



X 


X 

amortization 

2e 








amphitheatre 

1 

-.023 







amusement park 

1 

-1-.942 







anarchism 

l,2K.2e.S-.714 

X 

X 




z 

anarchists 






X 



ancestor 

1 

-.795 





X 

X 

ancestor worship 


...... 






X 

anchorage 








X 

ancient 

i 

-I-.647 






z 

animal husbandry 

1 

-.828 







animate 

1 

-.211 






, 

animism 

1 

-1.450 



. , X X 

X 



animosity 

1 

-.128 






, 

annals 

1 

-.335 






X 

annexation 

1, s 

-.341 

X 

X 




z 

annihilate 

1 

-.180 






X 

annual 

• t * » 







X 

annul 

1 

-.102 

X 



X 


X 

anointment 

1 

-.787 

X 

X 

X . . . 




anonymity 

1 

-1.144 







antagonism 

1, 2s, E 

+ .247 







anthropology 






X 



anthropoid 

.... 




X X 

X 



anthropomorphism 

2s 




X 




anticipation 

E 








anti-federalist 

S 








antipathy 

1 

-.453 







anti (as prefix) 

1 

+ .352 







anti-slavery 

.... 







z 

anti-social 

1 

-.450 



X X 

. X 



antiquity 

1 

+ .199 






X 

anti-trust laws 

1 

-.152 

X 



X 


X 

apartment house 

1 

+ .794 






.. 

apology 

1 

+ .673 







apostle 

1 

-.089 







appanage 

1 

-.152 







appeal , 

2g, S 


X 



X 


y 

appeal to the country .... 





X 



appellate jurisdiction 

1. 2g 

-1.14 

X 

X 


• 


X 

application 



- 

• 


X 
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Word 


1 

E 

tfl 


a 

M 

i 


Wesley Tests, 

Wesley 

College 


Editions No. 

e 

o 



.A 

C 

o 

£ 

^ 

3 

a M 

s 


NN M ^ 

' eu 


& 


appoint 



appointee 

1. s 

-.063 

appointment 

1. 2gi S 

+ .747 

apportion 
apportionment of 

1. S 

+ .395 

representation 

2g 


appraiser 

2g 

+ ..505 

appreciation 

1 

apprehension 

1 

+ .0,58 

apprentice 

1 

-.238 

approach 

E 


approbation 

2s 


"appropriate Iegislation”2g 


appropriations 

l,2g,2s,S4-.300 

approval 

2s 


arbitrage 

2e 


arbitrary 

1 

-.oii 

arbitration 

1.2g, S, 2e+.]22 

archbishop 

1 

-.477 

archduke 

1 

-.597 

architecture 

1 

+ .321 

archives 

1 

-.792 

archon 

1 

-1.34 

area 

S 


arena 

1 

+ .099 

argument 

1 

+ .616 

aristocracy 

1, 2g 

+ .222 

aristocrat 



armada 

. 


armament 

1.2g 

+ .179 

armed neutrality 

1 

-.792 

armistice 

i,s 

+ .111 

arms 



army 

S 


armor 



arrears 

i 

-.275 

arraignment 

* . . . 


arrest 

s 


arrogance 

1 

-.005 

arsenal 

1 

+ .205 

arteries of traffic 

1 

+ .057 

art gallery 

1 

+ .532 

article 


articles 

s 



X 


X X 


X 


X 


X 


z 


X 


X 


X 


X 


z 


zz 

z 

X 

■ 

X 


X 


X 

X 

z 


X 

X 

X 


z 

X 
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E 

X 

W 

M 
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(A 

Wesley Tests, 
Editions No. 

College 


§ 

e 

O 

s 

s 

AI 

III 

Mil 

11 

I 

Unpub, 

Pub. 

Kelty 

Pressey 


articles of 

Confederation 

articulation 

artifacts 

artillery 

artisan 

art of living togethi 
art 

ascetic 

asceticisrn 

ascendency 

aspersion 

aspiration 

assailant 

assassination 

assault 

assembly 

assent 

assertion 

assess 

assessment 

assessor 

assets 

assign 

assignment 

assimilate 

assize 

associate 

association 

assumpsit 

assumption 

assurance 

asylum 

atavism 

atheist 

at large, delegate oi 
representative 
atomization 
atonement 
atrociousness 
atrocities 
attache 
attack 
attainder 


2g. S 


2s 


i’ ' ' 

+ '.579 

1 

-.269 

2s 


2s, 1 

-.085 

1 

-.882 

2s 


E 


1 

-1.077 

1 

+ .189 

1 

+ .288 

1 

+ .658 

i.2g,S 

+ 779 

1 

+ .453 

1 

+ .505 

1, 2g, 2e, S 

+ '.237 

1. 2g 

+ .253 

1, 2e, S 

-.005 

1 

-.075 

i, 2s, E 

-.012 

1 

-1.319 

I 

+ .421 

1, S, E 

+ .616 

2g 


1 

+ .132 

1 

+ .242 

i.s 

+ .427 

i"' 

-.413 

S, 2g 


E 


1 

-.755 

.... 

+!ii6 

S,2e 


S, 2g 



X 


X 

X 

X 

X 


X 

X 

X 


X 


X 


X X 


X X 


X 


X X 


X X 


X X 
X X 


X 

X 


X 

X 



X 


z 


X 


X 


X 


X 


X 


X 


X 


X 

X 

y 

X 
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List Symbol 
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Wesley List 

Wesley Tests, 
Editions No. 

Wesley 

College 

i ^ ^ 

a 3 u 
P ^ 

& - 


NH 

H > 

pm Pm 

attendant 

1 

-f-.S58 







attitude 

1. 2s, E 

H-.4S3 





X 

, 

attorney 

2g, S 






X 

X 

attorney-general 







X 

X 

auditor 

1, 2g, 2c, S 

+ .146 

X 

X 


X X 

• • * 

X 

augment 

1 

-.178 







auspices 

1 

-.063 





. . 


Australian ballot 

1, S, 2g 

+ .644 

X 




X 


authentic 








X 

authoritative statements . .. 






X 


authorities 







. 

z 

authority 

1, 2s, E, S 

+ .S53 

X 




• > • 


authorize 

1 

+ .4S3 

X 




< • * 

z 

autocracy 

1. 2g 

-.257 

X 

X 

X 


X 

X 

autocrat 

1 

-.115 

X 




X 

X 

autogeny 

2$ 






. 


autonomic behavior 

.... 






X 


autonomous 







X 


autonomy 

i 

-.919 

X 

X 

X X 

X X 


X 

auxiliary 

1 

-.136 





X 

, 

aviation 

1 

+ .847 






X 

avocation 

1 

-.655 







axiological 

E 







, 

bachelor of arts 

1 

-.375 






. 

backward communities .... 






X 

« 

backward country 







X 

• 

backward regions 

1 

-.285 






• 

backwoodsman 

1 

+ .421 






a 

bail 

1, 2g, S 

+ .121 

X 

X 


X 


X 

bailiff 

1 , s 

-.865 

X 





X 

balance of power 

i.s 

-.923 

X 




X 

X 

balance of trade 

1, 2e 

-1.03 






• 

balance sheet 

2c 








balance, social 

2s, E 







X 

ballot 

2g, S 








ballot box 

2g 







a 

ban 

1 

-.293 





• ^ • 

a 

band 







. - . 

X 

bandit 








X 

banish 

i 

+ .474 

X 




X 

X 

bank 

2e 







y 

bank, banker’s 

1, 2e 

-1.12 




• ■ 



banker 






t * 

X 


bank, Federal Reserve 2e 





• 


■ 
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Word 



Wedey 

Wcrfey Test*, College 
Editions No. - — • — ■ 



bank reserve 

bankrupt 

bankruptcy 

bank solvency 

barbarian 

barbarism 

bargaining power 

barge 

barons 

barracks 

barrage 

barricade 

barriers 

barrister 

barter 

base of operations 
basilica 

basis of mutual aid 
battalion 
batteries 
battle 
battleship 
bavonet 
behavior 
behavior pattern 
beliefs 
believers 
bellicose 
^>elligerent 
benefactor 
bench 
beneficiary 
benefits 

benefit of clergy 

benevolence 

benevolent despotism 

bench (King’s) 

bequest 

besiege 

bestow 

betray ' 



bicameral legislature 


1 

-.660 


i.'2e 

-i;479 


1 

-1.097 


1 

+ .042 


1, 2s 

+ .025 


1 

-.727 


1 

+ .016 


1 

-.158 


1 

+ .379 


1 

-Oil 


1 

+ .226 


1 

+ .153 


i'" 

+ .274 


1 

-.148 


1 

-1.187 


i”' 

-.280 


1 

+ .326 


i‘ " 

+ .SIS 


2s 



1, 2s, E 

-1.732 


1 

+ .226 


1 

+ .226 


i,' 2 g. s 

+ .053 

X 

1 

+ .311 


i ' 

+ .616 


1 

-1.138 

X 

1 

-1.60 


1 

-1.019 

X 

1 

-1.386 

X 

1 

-1.437 


1 

+ .642 


1 

+ .279 


1 

+ .642 


1 

-.887 

X 

1 

+ .500 

. 

l.s 

-,914 

X 
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Word 

List Symbol 

Morton Weights 

m 

Wesley Tests, 
Editions No. 

Wesley 

College 

S? 

O. .A ± g 

B .3 ^ E 

P E. Ml A. 

II 

I 

III 

IV 

bidder 

1 

+ 357 





bids 






X 

bigamy 

1 

+ 190 




X . , . 

big business 

1 

- 737 




• • • • 

bigotry 

1 

- 324 




» > f • 

bill 

1, 2g, 2e, S+ .516 

X 



X z 

bill board 

1 

+ 869 





bill of attainder 

1 

-1 003 

X 



. 

bill of exchange 






X 

Bill of Rights 

1, 2g. S 

- 697 

X 



• • • • 

bill drafting bureau 

2g 


, 




bimetallism 

2e 

, 




X X 

biology 



. 


. 

X 

bi-partisan 



X 



* • * 

birth certificate 

1 

+ 262 


, . 


• • • • 

birth rates 

1 

- 149 





bishop 

1 

+ 237 




X 

blacklist 

1 

- 133 


« . 

, , 

X 

blackmail 

1 

-f 005 





blasphemy 

1 

- 133 





bloc 

1 

- 851 

X 

, 


X 

blockade 

1 

+ 184 

* 



. . z 

blockade runner 

1 

+ 173 

. 



• • ■ 

block house 

1 

+ 237 



, 

* • 4 ■ 

blue laws 

1 

- 233 

X 



■ > . s 

blue sky laws 





. 

X . . , 

bluster 

1 

+ 374 


. 


• • > . 

board 

Zg, h 




X X 

• • . y 

board of directors 

1 

+ 137 


. 

X 

• 4 

boast 

1 

+ 721 

, 

. 


• • • • 

body 

s 





X 

bodyguard 

1 

+ 584 


. 


X 

body politic 

1 

- 854 

X 



X 

bogey 

1 

- 005 




X 

bogus 






X 

Bolshevist 

1. 2e 

- 369 

X 



X 

bombardment 

1 

+ 406 




zz 

bombs 






X 

bonanza 

1 

- 714 




• • • • 

bond 

I, 2e, S 

+ 047 



X X 

X jr.h.s. z 

bondage 

1 

+ 316 

X 



X 

bonding company 






X . . . 

bondholder 

1.2c 

+ 132 



X 

X 

bond house 


• • ■ • 

• 



X 
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•8 

M 

u 
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a 

§ 

e 

% 

4> 

"3 

AI 

III 

II 

I 

Unpub. 

Pub. 

Kelty 

Pressey 


bond issue 

i,s 

-.190 

X 









bondsman 











X 

bonus 

i, 2e 

-.012 

X 







X 

X 

bonus system 

1 

-.S82 










bookkeeper 

1 

+ .779 










bookkeeping 

boom 

.... 


• 







X 

X 

bootlegging 

i 

+ .942 

X 

X 







X 

booty 

1 

+ .189 










border 











X 

borough 

1. 2g 

-.602 

X 

X 

X 


X 




X 

boss, political 

1. 2g, S 

+ .811 

X 




X 

X 


X 

X 

bounaaries 

i.s 

+ .826 

X 








z 

bounties 

1 

-.138 









X 

bounty, export 

2e 

-.893 










bourgeoisie 

1 

X 

X 

X 


X 

X 



X 

boycott 

I.S 

-.328 








X 

X 

Brahman priest 
branding 

i" 

+ .305 








; 5 

X 

branch 

s 










X 

brassage 

2e 

+ .137 





X 





breach 

1 

X 








X 

breach loading rifle 

1 

-.556 









. 

bread winner 

1 

+ .663 









X 

breastwork 











X 

bribe (v.) 

I.S 

+ .563 

X 

X 






X ir.h.s. . 

bribery 

l.s 

+ .506 

X 








z 

'bridge 

brief 

1 

+ .837 









X 

brig 

• • ■ • 










X 

brigade 

1 

-.191 









X 

brigadier 

• • • • 










X 

brigands 

1 

+ .115 









X 

broadside 

< * . 1 










X 

broker 

1, 2e, S 

-.201 

« 







X 

X 

bronze 

1 

+ .231 










brotherhood 

1 

+ .495 










browbeat 

1 

+ .252 










brutalities 

1 

+ .595 









, 

buccaneer ' 

1 

-.719 

X 







X 

X 

budgetary reforms 

i,2g,S 









X 


budget 

-.286 

X 

X 

X 

X 

X 

X 



z 

buffer state 

1 

-.719 

X 







X 

X 

building code 

1 

-.713 

• 







• 

• 
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Word 

List Symbol 

Morton Weights 

Wesley List 


Wesley Tests, 
Editions No. 

Wesley 

College 

3 . X S 

c S 

D bd dH 


II 

Uy4 

III 

IV 

campus 

1 

•f .737 





canal 






5 X 

canalization 

2t, 






cancel 

1 

+ .721 





cancellation of debt 

1 

-.017 





cancelled check 

1 

+ .179 





candidate 

1, 2g, S 

+ .647 

X 

X 


X z 

cannibalism 

1 

+ .516 




. . 6 . 

cannon 

. • • • 





4 X 

cannonade 






, , . X 

canon 

1 

-.691 




. . X 

canon law 

1 

-1.06 

X 

X 


X 

canonize 

1 

-.939 




• r • i 

canton 

1 

-1.06 

X 

X 

X . X . 


canvassing 






X 

capital (not econ.) 



X 



t 

capital (econ.) 

i, 2e, S 

-.375 

X 


, . X X 

X . 7. 

capital goods 






X 

capitalism 

i. 2e 

-.475 





capitalist 

1 

-.074 





capitalization 

2e 






capital punishment 

I, 2g 

+ .058 

X 

X 


X 

capital ship 

1 

-.864 





capitation tax 

1 

-1.25 

X 

X 

X X X X 

. X 

capitol (pol.) 

1. s 

+ .348 

X 



X 

capitulary 

1 

-1.46 





capitulate 

1 

-.556 





captain 






X 

captain of industry 

1 

-.074 





captivate 

1 

-.012 





captive 

.... 





X 

capture 






X 

caravan 

i 

+ .316 




X 

cardinal 

1 

-.347 




X 

career 

1 

+ .574 




♦ . . • 

car^ 

.... 





X 

cancature 

1 

+ .063 





cars and locomotives 






X 

carnage 

i 

-.175 




. « • • 

carpet-bagger 






. . . X 

cartoon 

1 

+ .705 




. jr.h.s. . 

case 

.... 





. . . y 

case method 

1,S 

-1.167 




» • 

case study 

1 1 ' 





X X 

X 
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Word 


m 

U 

w 

•• 

Weiley Teiu, 
Editioni No. 

Wesley 

College 


c 

o 

o 

s 

>x 

•a 

AI 

111 

Mil 

11 

I 

Unpub- 

Pub. 

1 ^ 

t>d cu 


case worker 

1 

-I 167 


cashier 
cash on hand 
cast 
castle 

1 

+ 5S8 


careermen 

2g 



casualty 

catastrophe 

1 

•f 38S 


catechism 

1 

- 339 


categories 

cathedral 

1 

- 032 


Catholicism 

1 

+ 157 


catholic orders 

1 

- 587 


caucus 

1. 2g, S 

- 533 

X 

causation (social, 
natural) 

2s, E 



cause and effect 
cavalier 

1 

- 612 

• 

cavalry 

cede 

1 

- 195 

X 

celebration 

1 

-f- 563 


celibacy 

1 

- 692 


censor 

censorship 

1 

+ 442 

X 

censure 

1 

4" 385 

X 

census 

1. 2g, S 

+ 516 

X 

census enumerators 
center of influence 

1 

-.633 


center of population 

1 

- 211 


centers 

centralization 

I, 2s 

- 084 

X 

centralize 

1 

-.021 

X 

central government 

1 

+ .169 

X 

century 

, , 


. 

ceremonialism 

2s 



ceremony 

1, 2s 

+ 474 


certificate 

1 

+ 574 

X 

certificate of ownership 2e 



certification of election 

2g 

. . 


certify 

... 

X 

cession 

1 

-.004 

X 

chagrin 

1 

+ 079 


chain stores 

1, 2c 

+ 853 

, 

chain units 



- 


X 

X 


X 

X 


X 


X 

X X z 


X 

I 


X 


X 

% 


X 


X 


X 

X 


X 


X 


t 


t 


z 


5 X 


[ 515 ] 


M H 



TESTS AND MEASUREMENTS 




«• 

% 


Wesley 




I 

Wesley Tests, 

College 





Editions No. 




Word 

E 












1 ^ 


JO 

3 


S 


S 

U ^ 

O 4> 

S ^ 

I 

II 

IIK 

III 

IV 

& -O 

1.5 ^ 

0- 


w 

U 

u 

cu 

chairman 

1.2g 

+ .674 X 

X . . . . 


X 

X 

chamber 

Chamber of Commerce 

+ .679 . 

X 



X 

i. 2g, 2e 

chamberlain 




X 

chancellor 

i 

-.677 X 




X 

chancery 
change of venue 

2k 



X 


X 

changing 

chaoi 

i ' 

+ .068 . 


X 



chapel 

1 

+ .S0S . 



6 


chaplain 

characterittic 

2g 



X 



charge (econ.) 

1 

+ .i68 . 





charges (legal) 
charges d’afTaires 

i ' 

-1.056 X 




X 

X 

charity 

1, 2s, S 

+ .616 . 


X 

4 

X 

chariot 

1 

+ .600 . 





charlatans 

1 

-1.018 . 





chart 

1 

+ .584 . 





charter 

1. 2g, S 

+ .122 X 

X 

X 


z 

chastity 

chateau 

1 

+ .126 . 




X 

chattel 

i 

-.570 . 





chauffeur 

1 

+ .753 . 





Chautauqua 

1 

+ .194 . 





chauvinism 

I 

-1.155 . 





cheap money 
check 

1 

2e. S 

-.977 . 

X 

X 



check, certified 

2e 


X 




checking account 

1 

+ .222 . 

X 




checkmate 
checks and balances, 

1 

-.426 . 

. 




System of 

1, 2g, S 

bo 


X 


X 

chevalier 

1 

-.998 . 





chief executive 

1 

+ .185 X 





chief justice 

2g 

X 




X 

chieftain 




X 



child guidance 

i 

-1.119 . 





child labor 

1, 2e 

+ .047 . 





child labor laws 
child welfare 

2e 

1 

-.084 . 

■ • ■ * * 


X 


Chinese 




X 



chivalry 

1 

-.085 . 





ci^ret 

1 

+ .511 . 
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Weiley Teit», 
Edition! No. 


Weiley 

College 


JO 

3 

& 


^ S' 

S S S C D Pu 


I J 


christianize 

1 

+ 395 



christen 

1 

- 015 



chronicle 

1 

- 090 



chronological age 
church 

i,s 

-!- 795 



church courts 

churchman 

circuit-court 

1 

2g.S 

-t- 237 

X 

X 

circuit-judge 

1 

- 459 

X 


circulating capital 
circulating medtum 

1 , s 

- 834 

• 


circumstances 

1 

-f- 442 

, 


circumnavigation 

1 

- 054 



citadel 

city 

city council 

1 

2g,S 

1 

- 174 

-h 437 

X 


city manager 

li 2g, S 

-1- 037 

X 


city planning 

1, 2k, S 

- 084 



city social welfare 
activities 

2g 

- 528 



city state 

1 

X 

X 

citizen 

1, 2g, S 

+ 432 

X 


citizen army 

1 

- 089 



citizenship 

2g, S 

- 348 



civic 

l.S 

X 


civic center 

S 




civil 

civil case 

1 

- 655 

X 


civil government 

1 

- 158 

X 


civilian 

1 

- 053 

X 

X 

civil law 

1.2g 

-.628 

X 


civil liberty 

1 

— 666 

X 


dvil marriage 

1 

■“ .608 

X 


dvil rights 

1, 2g 

-.714 

X 


dvil service 

1. 2g. S 

+ 058 

X 


dvil struggle 

1 

-.748 



dvilization 

i,s 

-t-.179 

X 


civilized 

s 

-f 278 



claimant 

1 

X 


claims 

1 

-t- 579 



claims adjuster 

1 

- 882 



dan 

1 

-.258 

X 


clash 

dass consciousness 

1, 2s 

- 770 

* 
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Word 


3 

E 

X 

m 

m 


Wesley 

Wesley Tests, College 
Editions No - — • — 



3 

(X 


dattes 




classic 




classical 




dasstlied service 

2g 



class legislation 

1 

- 804 

X 

dass struggle 

1, 2s 

- 661 


clause 

1 

+ 183 

X 

deanng house 

2e 



cleavage 

1 

- 265 


demency 




der^ 

1 

4- 373 


clericalism 

1 

- 902 


derk 

S 



clerk of court 

2g 



clerk of the House 

2g 


. 

dient 

1 

+ 2S3 

X 

clientele 

1 

- 559 


climate 




clinic 

S 



clique 

1 

- 012 

X 

closed season 




dosed shop 

1, s 

-.517 


closure 

2g 



clothing 




cloture 




club 

S 

. 


co-adaptation 

2s, E 



coaling station 

1 

-t- 195 


coalition 

2g 

- 554 

X 

coast 




coast guard 

2g 



coasting vessel 

1 

- 248 


code 

1 

+ 048 

X 

codify 

l,2g 

-.485 

X 

coerce 

1 

- 236 


cohesion, social 

2s 

. 


coinage 

2e, 2s, 2g, S 



collaborate 

1 

- 368 


collateral 

l.S 

- 481 


colleague 

1 

-f- 132 

X 

collections between 




banlu 

2e 



collective 

1 

+ 069 


ctdlective bargaining 

1, 2e 

-.814 




X 

X 

X 


X 

X X 

X 

jr h s t 

, . X 



X y 
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Word 

J 

E 

M 

X 

M 

s 

4^ 

Wesley Tests, 
Editions No. 


Wesley 

College 



M 

c 

S 

o 

s 





> 

Unpub. 

Pub. 

•V 

1 

o. 

collective ownership 

1 

-.655 









collectivism 

2e, 2s 










collector 

S 


X 








college 










X 

colonel 










X 

colonial policy 

1 

-.si6 

X 








colonies 

1. 2g, S 

+ .416 

X 





X 


s 

“the color line" 

1 

-.344 









co-maltet 








X 



combat 










X 

combination 

2s, S, E 









X 

commandeer 

1 

-047 









commander 










y 

commander-ln-chief 

i.2g 

+ .616 

X 

X 







commensurate 

1 

-.661 









commerce 

l.s 

+ .558 








z 

"commerce clause” 

2g 







• • 



commercial banking 

1 

-.677 






* . 



commercial interests 

1 

-.339 









commercial regulation 

1 

-.429 









commercial rivalry 

1 

-.306 









commercialization 

2s 

-.859 









commissariat 

1 








z 

commission government 1, 2g, S 

,751 

X 

X 







commission merchant 

2e 


e 



X 




• 

commissioner 

commissioner 

2g 


X 





X 



extraordinary 


-.206 






X 


, 

commissions (army) 

i 

X 







• 

commissions (group) 

1 

-.379 

- 







r 

commit 



. 







X 

committee 

Committee of the 

1. 2g, S 

+ .558 

X 

X 




X 

z 

zz 

Whole 

1. 2g 

-.912 

X 







. 

commodities 

l.s 

+ .384 








z 

common carrier 

l.s 

-.491 

X 

X 

X 





. 

common council 

2g 


« 







. 

common defense 

2g 









• 

commoner 


-.593 

- 







X 

common lands 

1 








. 

common law 

1. 2g, S 

-.570 

X 

X 




X 


X 

common people 

2s 


' 







X 

common purpose 


■ 







- 

commons 

1 

— .544 



• • 



• • 


X 
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Word 

List Symbol 

s 

M 

3 

•» 


Wesley Tests, 
Editions No. 

Wesley 

College 



e 

§ 

o 

s 

M 



X 

III 

IV 

Unpub. 

Pub. 

Kelty 

Pressey 

Common! (House of) 

S 










common stock 








X 



commonwealth 

i. 2g. S 

-.232 

X 

X 

X 


X 



zz 

commune- 

1 

-1.088 

X 








communication 

1, 2s, S, E 

+ .495 

X 




X 

X 


z 

communion 

1 

+ .052 









Communion Supper 

1 

+ .062 









communism 

I,2s,2g,E 

-.527 

X 

X 



X X 

X 


z 

communist 

1 

-.548 

X 





X 



community l,2g,2s,S,E 

+ .300 

X 







z 

community center 

i.s 

+ .205 









community chest 

1 

+ .105 






X 



community enterprise 








X 



community fund 

.... 







X 


, 

commutation 

.... 







X 



commute 

I.S 

-.063 









compact 

1. 2g 

+ .247 

X 







X 

companionship 

2$ 







, , 


, 

company 

S 

-.185 








z 

compatriots 

1 

X 





, , 


, 

compensation 

compensation 

l,2s,S,£ 

+ .311 






X 


X 

mechanism 

£ 










competition 

l,2s,2e,S 

+ .395 






X 

X 

z 

complainant 
complex (cultural, 

2s, E 








• 


X 

social) 

composition (demotic, 







X 


• 

social) 

2s 








X 

z 

compromise 

1, 2s, S, E 

+ .437 

X 

X 






X 

comptroller 

1. 2g, S 

-.861 

X 

X 

X 

X 

X 




compulsory 

1 

+ .553 









compulsory arbitration 

1 

-.525 

X 




X 




compulsory education 

1 

+ .426 









concatenated 

.... 







X 



compurgation 

1 

-1.402 









concede 

«... 









zz 

concentration 

1 

+ .368 









concern 

. . . • 









X 

concert 

1 

+ .616 









concerted yolition 

E 









. 

concesuon 

1 

+ .142 






X 


X 

conciliate 

1 

+ .201 






• 


zz 
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Word 


conciliation 

conclave 

concord 

concordat 

concurrent 

concurrent Jurisdiction 

condemn 

confederates 

confederation 

Confederacy 

conference 

conference committee 

confession 

confidence 

confirm 

confirmation 

confiscate 

confiscatory 

conflict (v.) 

conflict, situation 

conflicting 

conform 

congeries 

conglomeration 

congregation 

Congress 

Congressional Record 

congressional district 

congressman 

conjoint 

conjugation 

connive 

conquer 

conquest 

consanguinity 

conscience 

consciousness, self 

consciousness, social 

consciousness of kind 

conscription 

consecration 

consensus 

conservation 

conservative 


List Symbol 

Morton Weights 

Wesley Lilt 


Wesley Tests, 
Editions No. 

« ^ S > 
*1^ 

Wesley 

College 

j£ 

3 

D a. 

fr 

T3 

U 


1. 2g 

+ .132 

X 


. 




X 


* 







• « 




z 

i 

-.oi 2 










1 

-1.145 

X 

X 



• « 

* 



X 

s 


X 

X 

X 


X 

* 



z 

.... 








z 













z 

i 

+ .242 

X 








zz 

1, 2s, 2g, 

S+.158 

X 








X 

1.2g 

+ .163 

X 








z 

1. 2g, S 

+ .553 

X 







X 

z 

2g 






• « 





1 

+ .484 

X 




• * 





1 

+ .642 





• • 





1 

+ .379 

X 




• • 





1 

+ .083 

X 




» » 


X 



1, 2e 

+ .258 

X 




X 

• 



zz 








» 

X 



i, 2s, E 

+ .526 






• 


• 


E 


X 

X 






s 

z 

1 

+ .632 








• 

» 

1 

+ .358 









» 

1 

-1.797 








» 

* 

1 

-.487 








• 


1, 2s, E 

+ .426 








■ 

X 

L 2g, S 

+ .558 

X 




X 




zz 

2g 









■ 

• 









z 

t 

s 

2g,S 









• 

s 

1 

-1.181 










2s 











1 

-.592 



















z 

X 

i, 2s 

+ 553 









z 







X X 



• 

• 

1 

+ .332 








« 


2b 









• 

• 

2b, E 










p 

2s, E 











• 

L 2g, S 

+ .195 

X 

X 







X 

1 

+ .064 








s 

• 

2s, E 









• 

- 

L 2g, S 

+ .068 







z 

« 

* 

LS 

+ .011 

X 

- 

• 


« • 

• 

X 

• 

X 
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m 




•& 


Word 

1 

E 

i 

« 

Z3 


J? 

G 

B 




w 

0 

:? 



s 



Edidona No. 


Wesley 

College 


§• •§ -5 5 




conservatism 

1, 2s, 2g 

\ 

P 

X 



. 


consignment 







X 

consol 

2g 








consistent mode of 
response 





X 


consolidated school 

i 

-.016 






consolidation 

1, 2e, 2s, S, E+.005 

X 




X 

conspiracies 

1 

+ .321 

X 

X X . X . 


, , 

z 

conspirators 

1 

+ .384 

X 





conspire 

1 

+ .316 






constable 

1. 2g, S 

+ .358 

X 




X 

constabulary 

1. 2g. S 

-.353 

X 



* 


constancy 

1 

+ .195 




. jr.h.s. 


constituency 

1. 2g 

-.458 

X 

X X X X X 


, 


constituents 








constitution 

1> 2g 

+ .395 

X 



. 4 

z 

constitutional 

« * « * 





X 


constitutional 

amendments 






X 


constitutional basis 

• • • . 





X 


constitutional 

monarchy 

i.s 

-.449 

X 





constitutional revision 





X 


constitutional 

supremacy 






X 


construction (loose, 
strict) 

l,2g 

+ .463 






consul 

1. 2g 

-.200 

X 




z 

consul ^neral 






X 


consul indorsement 

• 9 » * 





X 


consular service 

1 

-.405 

X 

X X , X . 




consulate 

1 

-.274 

X 



• • 


consumer 

... 4 





X 

z 

consumption 

1, 2e, S 

+ .489 

. 


X 

* » 


consumpdon tax 

1 

-.769 

X 



• • 


contact 

2s, E 


. 



• • 


contagion 

2s 





• ■ 


contagion, social 

E 





* • 


contagious disease 

. . . « 


• 


X 

» • 


contemporary 

1 

+ .225 

• 



• • 


contemporaries 

i ' 





• 

X 

contempts 

+ .6i6 

. 





contempt of court 

1> 2g 

-.580 

X 



X 


contentious 

.... 


• 



X 
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Word 


i 

E 


m 

JC 


Weiley 

*5 


Weiley Testi, College 


ti 

Editioni No. - — ^ ^ 

c 

o 


ji 

C 

o 

s 

■a 

V 

I 

II 

UK 

III 

IV 

Unpu 

Pub. 


3 


contest 

continent 

continental 

1 

1 

-l-.Sll 
+ .322 








■ - 


X 

X 

X 

Continental Congress 

S 


X 










contingent 
continuity, social 

1 

2s, E 

-.702 








X 


X 

contraband 

1, 2g 

-.465 

X 

X 





X 



X 

contract 

2g, S 


X 







X 

• 

X 

contraction 










X 



contract labor 

i 

-.628 

X 

X 









contract, social 

2s 












contribution 

1 

+ .616 









6 


control, social 
control of industry by 

2s, E 












workers 

controller of the 

2e 











• 

currency 

2g 

+ .442 










• 

controversy 

1 

X 









• 

convene 

.... 










X 

■ 

convenience goods 

• « . . 









X 



convent 

1 

+ .421 











convention 

1, 2g, S 

+ .S47 

X 

X 






X 


X 

conventionality 

2s 











• 

convergence 

E 

+ ;542 










■ 

conversation 

1 









4 


conversion 

1 

+ .384 










X 

convey 

1 

+ .200 











convict (v.) 

1 

+ .484 

X 

X 








X 

convict (n.) 

1, 2g 

+ .553 

X 




X 

X 




y 

conviction (belieO 

1 , s 

+ .253 











convocation 

1 

— .665 











convoke 

1 

-.723 











convoy 

1 

-.539 










X 

cooperate 

1 

+ .516 

X 








X 


cooperation 

2e,2s,S,E 

-.454 








X 



cooperative marketing 

1, 2e 








X 

• 


cooperative societies 

1 

— .613 








- 


* 

co-ordinate 1 

co-ordinated governmental 

-.205 







• 

• 


• 

agencies 









• 

X 


• 

co-ordination . . . ■ 

co-partnership of capital 








• 

X 


■ 

and labor 

2e 

-.122 








• 

■ 

• 

cope 

1 








■ 

« 

• 
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Word 

1 

1 

W 

M 

Morton Weights 

Wesley List 


Wesley Tests, 
Editions No. 

Wesley 

College 

a 

o. ^ 

Kelty 

Prcssey 


X 

AI 

III 

copperhead 






. . 




X 

copyright 

i, 2 g, s 

-.122 

X 



X X 


X 

z 

X 

cordon 

1 

-.713 









coronation 

1 

-1-.178 

X 







X 

coroner 

1. 2g. S 

+ .036 

X 







x 

corporal 

1 

+ .099 








X 

corporal punishment 

1 

+ .110 









corporate action 

2s, £ 










corporation 

1. s 

+ .i27 

X 



X X 


X 


zz 

corporation tax 

1.2g 

-.638 

X 








corps 

1 

-.268 









corpus delicti 

. . . t 







X 



correlary note 








X 



correlation 

i 

-1.007 









correspondence 

1 

+ .558 




X 





corrupt 

1 

+ .558 

X 





X 


z 

corsairs 

1 

-.684 









corvee 

1 

-1.360 









cosmopolitanism 

S, 1, 2s 

-.560 







• 

X 

cost belt 

1 

-1.333 






* 

• 


cost of production 

1, 2e 

-.260 







• 


cotton-gin 

1 

+ .532 









council 

1, 2g, S 

+ .547 

X 



X X 




z 

councillor 










x 

counsel 

2g 


X 


• • 




• 

X 

counselling 





« * 



X 

• 


counsellors 

i 

+ .368 

X 


• • 






counter-attack 

1 

+ .237 








X 

counter-drive 

, , 









X 

counterfeit 

1, 2g 

+ .663 








X 

counter-offensive 

1 

+ .058 









counterpoise 

1 

-.986 









countersign 

I 

+ .041 









country 

1, s 

+ .705 

X 





X 


y 

countryman 

1 

+ .590 









counts 

1 

+ .295 








X 

county 

2g,S 







X 


z 

county seat 

1 

+ .637 

X 



X X 





coup d'etat 

1. 2g 

-.823 

X 







X 

courage 

1 

+ .837 

, 






4 


courier 

1 

-.259 








* 

court 

1, 2g 

+ .558 

X 





X 


z 

court-martial 

1> 2g 

-.099 

X 

X 

X 

X 


X 


X 

court of appeal 

— 


• 

* 

• 

• 


• 


X 
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APPENDIX 11 


Word 

List Symbol 

Morton Weights 

Wesley List 

Wesley Tests, 
Editions No. 

^ 

M M MW ^ SiM 

Wesley 

College 

a 

O- kO 

S Ok 

court of records 






X ' 


courtesy 

1 

+ .721 



• ■ 



courtiers 

1 

+ .016 




. 

X 

covenant 

1 

-.374 





X 

covet 

1 

+ .2S3 






covetous 






X 


cowboy 







5 . 

craft 

1 

+ .3i6 





« 

craftsman 

1 

+ .068 




. 


creative art 

2s 







creative moment 

E 







credentials 

2g 


X 

X X 

X 


X 

credit 

1 , s 

+ .490 





z 

credit men 






X 


creditor 

i 

+ .306 



, • 


« * 

credit structure 





• * 

X 


creed 

i 

+ .i78 



» • 

X 

x z 

creole 

1 




• » 


X 

crew 





• • 


4 X 

crime 

i, 2s, s 

+ .6i6 

X 

X X 

1 • 

, » 

z 

crime wave 

1 

+ .183 



t • 


, , 

criminal 

l.s 

+ .553 

X 



X 

X 

criminal law 

1 

-.096 

X 


• • 


, , 

criminal offense 

1 

-.917 

X 


• • 



criminology 

1 

-.402 



• • 

x 


crisis 

1,E 

+ .384 



• ■ 

• . 

z 

criterion 

1 

-.706 



• • 

• , 

. • 

critic 

1 

+ .321 



1 • 

. . 

jr.h.t. . 

criticism 

1 

+ .505 



1 • 

. 

• 

crop 





* • 

. 

X 

cross fertilization of 








culture 

2s 




• • 

• 

• 

cross questioning 





r • 

X 

• • 

crowd 

E 




• • 

. 

• 

crown 





k • 

- 

X 

crown prince 

i 

+ .384 



• • 

. 

• 

crucial situation 

E 




• • 

• 

• 

cruisers 

1 

+ .174 



* “ 

a 

zz 

crusades 

1 

-.137 



• • 

- 

ii.h.s. z 

crystalization 

1 

-.278 



■ • 

• 

• 

cue 

2s 

+ .131 



• • 

• 


culprit 

1 

+ .453 



■ • 

■ 


cult 

1 

— .604 



• a 

x 

X 

cultivation 

1 

+ .574 


• 

• 

• 

s t 
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Word 

2 

s 

if 

“C 

m 


Wesley Tests, 
Editions No. 

Wesley 

College 

t 

cn 

«• 

e 

§ 

o 

s 



AI 

III 

Mil 

II 

Unpub. 

Pub. 

Kelty 

Pressey 

cultural lag 






X . . . 

cultural itability 






X 

culture 

i, 28, E 

+ .390 




X 

culture man 






X . » . 

cumulative preferred 
itocks 







cupidity 

i 

-.S35 




» • • ' 

curb 

1 

+ .500 




X 

cure 

1 

+ .748 





curfew 

1 

+ .179 

X 



. . 5 . 

curia 

< « « € 





X . t . 

curial 

1 

-1.423 



• • • • 


curiosity 

1, 2s 

+ .689 



• • • • 

. . 4 . 

currency 

1, 2e, S 

+ .358 

X 


. X 

X z 

current 

.... 






z 

current expenses 

...» 

-.165 




■ X • • 

curricula 

1 




* ■ • i 

curry 

1 

-.032 




# • • V 

curtail 

1 

+ .189 




• • • i 

custody 

1 

+ .184 

X 

X 

X , X . 

« ■ » 1 

custom (social) 

1, 2s, E 

+ .443 





customary codes 

2s 





« • • » 

customer 

1 

+ .689 




X Z X 

custom house 

1 

+ .127 




• • • « 

custom of the 
Constitution 

2g 





• • • • 

customs (econ.) 

1. 2g, S 

-.126 




Z 

customs duties 

1 

-.1995 

X 




cutthroat competition 

1 

-.282 





.-czar 

.... 





• • » X 

cytology 

.... 

-1.781 





Daimio 

1 




• V » * 

dairy 

1 

+ .674 




. . 4 . 

damage 

1 

+ .608 




• • • ■* 

damage suit 

1 

+ .352 

X 



■ • • • 

damages, assess 

2g 





■ • • • 

Darwinism 

t • • • 






data 

1 

+ .100 




• • • • 

. . 3r.h.8. . 

daunt 

1 

-.089 




day nursery 

1 

+ .153 




• • • • 

deacon 

1 

+ .179 



• • « ■ 

* • • V 

deadlock \ 

1 

+ .211 



V » • • 

. . . * 

deal 

« • « • 




• • • • 

. . a 

dean 

1 

+ .227 




• • • • 
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Word 

’«! 

List Symbol 

Morton Weights 

Wesley List 

Wesley Tests, 
Editions No. 

^ M M ^ 

Wesley 

College 

\ fr 

B 

eu ti2 

1 

Cu 

death penalty 

1 

+ .779 






death rate 

1 

+ .194 






debar 

1 

-.105 






debase 

1 

+ .011 






debate 







X 

debauchery 

i 

-.138 





, 

debenture, export 

2e 






. 

debt 

S 






zz 

debt-funding 

1.2g 

-1.014 

X 

» • a 




debtor 

«... 



• • ■ 


X 


debtor-class 

1 

-.466 


• • » 

X 



decade 




« • • 



z 

decadence 

2s, E 



• • • 




decentralize 

1 

-.248 





X 

decarchy 

1 

-1.674 

X 





decision 

1, 2g, S 

+ .6667 

X 


X X 

X 

z 

decisive 





X 

declaration 

Declaration of 

S 


• 




z 

Independence 

l,2g 

+ .850 

X 




• 

declaration of war 

1, 2g 

+ .733 

X 




. 

decorate (medal) 

1 

+ .567 


« « • 



, 

decree 

1 

+ .184 

X 




X 

dedication 

1 

+ .383 


• • > 


X 


deductive process 






X 

. 

deed 

S 






X 

de facto 

1.2g 

-.759 

X 




. 

defalcation 






X 

X 

default 

i 

-.029 





• 

defeat 

• • • • 






X 

defeatism 

I 

-1.136 





• 

defection 

1 

-.586 





* 

defectives 

1, 2s, S 

+ .362 



, 


• 

defendant 

1. 2g, S 

+ .267 

X 

X 


X 

X 

defense 

1. 2g, s 

+ .711 

X 

• • • 



X 

defense mechanism 

S 



« • • 



• 

defensive 

. « 1 • 



• • • 



z 

defiance 

1 

+ .711 



• • 

6 

• 

deficit 

1 

+ .223 



« • 


• 

definition of situation 

E 




• ■ 


* 

deflation 

• « • • 



» » • 



X 

deforesution 

.... 

+ .128 


» • • 


X 

• 

degeneration 

i, 2s, E 


• • • 


. . • 

• 

degradation 

1, 2s 

+ .201 


« ■ • 


• « • 
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II 

II>4 

III 

IV 

decree 

1 

+ .477 



» m • • 




deist 

1 

-.970 







de jure 

2g 


X 


» m t • 




delegate 

1, 2g. S 

+ .633 

X 


• • * • 


X 

z 

delegated power 

s 




• < > < 

X 



deliberate falsehood 





• m t • 

X 



deliberation 

1 

+ .372 







delimitation 

1 

-.614 



m » * • 




delinquent 

i.s 

+ .0S1 

X 


t m * • 



X 

deliverer 

1, 2s 

+ .511 



m » f • 




delusion 

1 

+ .211 



• ■ ■ 




demagogue 

i.s 

-.525 

X 

X 

X , X . 

X 



demaitoeuery 






X 



demands 

1 

+ .644 







demand, alternative 

2e 




• » * • 




deme-enrollment 

1 

+1.647 

X 






dementia praecox 

. • . « 




• * « • 

X 



demobilize 

1 

-Oil 






X 

democracy 

l, 2s, 2g, S 

+ .112 

X 

X 

X . . . 


z 


democrat 

1 

+ .439 

X 


« • < • 


z 


democratic 

.... 




• > • • 

X 



Democratic Party 

2g 




» r * * 




demolish 

1 

+ .572 



• » • • 




demonetization 

1 

-1.624 



• > « « 



X 

demonstration 

1 

+ .389 



» w * • 




demoralize 

1, 2s, E 

+ .256 



* * * » 

« * 



demote 

1 

-.007 



m » t • 

• * 



denationalization 

2s, E 




» m * • 




denomination 

l.E 

+ .364 



m 9 f t 



z 

denounce 

1 

+ .511 



9 t * • 




density of population 

1 

-.303 







department 

1, 2g, S 

+ .572 

X 





z 

department stoy 

1 

+ .744 







dependence 

S 







X 

dependency 

1, 2g, S 

-.028 

X 

X 


X 



dependent (s) 

1, 2s 

+ .589 







depopulation 

1 

-.107 



« V ■ ■ 




deportation 

1 

+ .045 

X 


• • • • 



X 

depose 

i,s 

+ .250 

X 





X 

deposition 

2g 




» • • • 




depository 

1, 2g 

.000 



« * * • 




deposits V 

1 

+ .506 



• • • • 

X 


X 

deposits as currency 

2e 




• » V • 




depreciation 

2e 




• • • • 



z 
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> 

Wesley 

College 

jd 

a 

o a 

:=> a. 

b 1 

3 J . 

depreciate 

1 

- 139 







depredations 

1 

+ 001 







depression 

1 

-1-.261 





xjt 

.h.t. X 

deprive 

1 

+ .511 

X . , r 






deputies 

1. s 

+ .294 

X . . . 



X 


x 

deputized militia 







X 


deputy-governor 

2g 








dervish 

1 

-.831 







descendant 

1 

+ .633 


X 



X 

X 

desecration 







X 


desertion 

i 

+ 633 

• • • • 





X 

desire 

E, 2s 


• • • • 



x 



desolate 

1 

+ .517 

» • » « 





6 , 

despoil 

1 

+ .107 






p • 

despotism 

1 

+ .121 

X . . . 





x 

destroyer 








X 

destitution 

i 

+ .322 


X 




s s 

deuchment 

E 


• • * • 





1 t 

detective 

.... 


« • • • 





X 

detention 

1 

+ .317 

• ♦ • • 





* s 

deterioration 

2s. E 


« • • » 





s s 

determinism 

1 

-.831 

. • • • 





• f 

deterrent 

1 

-.608 

• • • ♦ 





» ■ 

detestation 

1 

-.017 

. » . • 






dethrone 

1 

+ .306 

X « • • 





X 

detour 

1 

+ .772 

• « • • 






devastate 

I 

+ .322 

. • • • 






development 

1 

+ .506 







devolution 

. . , , 






X 


devotion 

1 

+ .711 






• p 

devout 

1 

+ .517 

• • • * 





t s 

diagnosis 

1 

+ .267 

* • • • 





p p 

dialects 

1 

-f- . l82 

» . • • 





a • 

dialogue 

1 

+ .339 

• • • • 





« a 

diary 

1 

+ . 517 

. . • » 





• a 

dictator 

1, 2g 

+ .106 

X z . ■ 





X 

dictum 



• • • • 




X 

« a 

difference of potential 

E 


. . • • 





« a 

differential 



« • * • 




X 

* a 

differentiation 

2s, E 








diffusion 

2s, E 


, . • • 





a a 

dignitaries 

1 

-Oil 

, • • » 





a a 

diligence 

1 

+ .456 

. . * • 





a a 

dimintshing returns 

1 

-1.292 

. . • • 


— ^ 

• 

X 

• a 
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S ^ 

^ « s a 


3 

a. 

> ^ 


diminishing utility 
diocese 
diplomscy 
diplomat 
diplomatic agents 
diplomatic corps 
diplomatic note 
diplomatic relations 
direct action 
direct election 
direct legislation 
direct marketing 
direct method 
direct primary 
direct sales 
direct taxes 
director 

directorate, interlocking 

directors, board of 

directory 

diridble 

disabilities 

disaffected 

disarmament 

disbanded 

discipline 

disconcerting 

discount 

discount of commercial 
paper 
discourse 
discover 
discredit 
discrimination 
disease 
disestablish 
disfranchise 
disinfectant 
disintegrate 
disintegration, social 
disloyalty 
dismemberment 
dismiss 

disorganization, social 


1 

1 

1, 2g, S 


292 . . 

874 . . 

079 X X 


338 X 


084 X . 

939 . . 

724 X . 

846 X 

945 . . 

840 X X 

072 . . 

761 X X 

289 X . 


179 X . 

oii X ! 

391 X X 
379 X . 
379 X , 
616 X X 
000 . . 


533 X . 
513 X X 


426 X 
074 X 


X X X 
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^ ^ l-N ^ 

Wesley 

College 

1 1 
c a ^ 

;=> 0. ud 

1 

cu 

dispatches 

1 

+ 311 

X 




X 

dispensary 

i.s 

- 058 





X 

dispensation 

1 

.506 

X 

X 



X 

dispersion 

disposal 

2s 

1 

+ 253 

• 





disposition 

I.S 

+ 437 






disputes 

I.S 

+ 568 

X 

X 


. . 4 

X 

disruption 

1 

+ 137 

X 





dissent 

1 

+ 121 





X 

dissolution 

1, 2s 

+ 016 

X 





dissolve 
distance, social 

1 

E 

+ 447 

X 




X 

distinction 

1 

+ 437 






distribute 

distribution, inequality 

l.E 

+ 495 






of 

2e 







distribution (wealth) 

1, 2e 

+ 036 






district 

1, 2g. S 

+ .558 

X 




y 

district attorney 

S 





. . 5 

X 

distrust 

1 

+ 568 






disunion 

1 

+ 231 

X 





divergence 

1 

+ 179 






diversification 

1, 2s 

- 264 






diversity 

1 

+ 085 






dividend 

1 

+ 264 




X 

X 

divine right 

1 

-.212 

X 





division 

2g 

, , 





X 

division of labor 

1, 2s, S 

- 201 




X 


division of spoils 

1 

- 195 

X 





divisions 

1. 2g 

+ 479 

X 





divorce 







X 

doctrine 

i 

+ i84 

, 



X 

z 

document 

1 

+ 253 

X 



X 

z 

dopfia 

dollars 

1 

-.628 





X 

i 

+ .705 






dollar diplomacy 

1 

- 729 

X 





domain 

1 

+ 015 

X 




X 

domestic 

1 , s 

+ 364 

X 




z 

domestication 

1, 2s 

- 005 






domestic manufactures 

1 

-.251 






domestic policy 

1 

-.300 

X 





domestic regulations 

1 

-.490 

X 




* 

domestic relations 

1 

+ 111 

X 


X 



domestic system 

1 

-.744 



. 


• 
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1 





c 



US t' 


C/3 

§ 

o 





M 

• #4 

V 

^ 

K S H S: 

c ■§ TS i 

D h ^ O. 

domeitic tranquillity 
dominance 

1 

-.316 

• 


X 

dominate 

i, 2s. E 

+ .368 




domineer 

1 

+ .021 




dominion 

1 

+ .201 

X 


zz 

dominions 

1 

+ .126 

X 



donation 

1 

+ .437 



X 

dormitories 

1 

+ .705 




dowager 

1 

-.436 




dower 

1 

-.760 




dowry 

1 

-.390 




downfall 

1 

+ .779 



. . 6 . 

draft (army) 

1. 2g, S 

+ .299 

X X 


5 z 

draft (money) 

1, 2e 

-.132 




draftee 

• • • • 




X . , , 

drafting 

dragoon 

i 

-.486 

X X 


X 

• • • f 

draining 

. . t . 





drawee 

dreadnought 

i ■ 

-.081 

X 


X . . , 

• • • • 

drug addict 
drummer 

1 

+ .068 



• « • a 

X 

dry 

dual control 

i " 

-.862 

X 


■ • ■ X 

duchess 

duchy 

i 

-.543 

X X 

X X . X 

X 

due process of law 

1. 2 g. s 

-.846 

X 



duel 

1 

+ .616 

. 


. . jr.h.s. . 

dues 

1 

+ .558 

• 


X 

'du^ut 

dukes 

.... 

+ .184 

X 


X 

z X 

duma 

1 

-1.035 

X 


• » • • 

dumping 

2e 


. 


• ■ • • 

dungeon 

duplication 

i ' 

+ .321 

• 


X 

• • • • 

duties 

hS 

+ .689 

X 


• • * • 

duties (revenue) 

1 

+ .132 

X 


• • • z 

dynamic 
dynamic forces 

E 


- 


•■4a 

3^ • 

dynamic immobility 



. 


X 

dynamic society 

i ' 

-.259 

. 


X 

dynasty ^ 

X X 

X X . X 

X 

eari 

1 

-.205 

• 


X 

Eastern Question 

1 

-1.009 

X 


. 
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eccentricity 
ecclesiastical 
ecological patterns 
ecology, human 
economic 
economic activity 
economical 
economic conditions 
economic conflict 
economic expansion 
economic factors 
economic imperialism 
economic independence 
of women 
economic 
interdependence 
economic laws . 
economic penetration 
economic rights 
economics 

economic stabilization 

economizing 

economy 

edicts 

edit 

editorial 

education 

education, agricultural 
education, technical 
educational 
qualifications 
educators 
efficiency 

efficient government 
eight-hour day 
elastic clause 
elastic currency 
election 

election, biennial 
electioneering 
election returns 
elective 

electoral college 
electoral district 


1 

- 474 


1 

- 318 

. 

E 



1 

+ 200 


1 

-.611 


1 

- 713 


1. 2e 

-.972 

X 

2e 



1 

-1 008 


2e' 

. 


1 

- 085 


1 

+ .137 


1 

-.126 

X 

1 

+ .106 


1 

+ .500 


1, 2s, S 
2e 

+ .501 


2e 

■ • 


1 

-.186 


1 

+ .501 


l.S 

+ .426 


1, 2e 

+ .443 


S 


X X 

1 

-.992 


1. 2g, S 

+ .632 

X 


^ a 

s s s 2: a 


2g 

1, 2g,S -.470 
1 -.407 
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c/3 

w 

M 

g 

g 



11 

111 

IV 

Unpub. 

Pub. 

Kelty 

Pressey 

electorate 

1, 2g, S 

-.115 

X 





electors 

2g, S 






X 

electrocute 






X 

eleemosynary 

institutions 






X 


elementary education 

1 

+ .422 






eligible 

i.s 

+ .437 

X 




X 

elite 

1 

-.549 






emanate 

1 

-.532 






emancipate 

1, s 

+ .363 

X 

X 

X X X X 


z 

emancipation of women 2s 






• . 

embargo 

1, 2g 

-.322 

X 

X 

X X . X 


z 

embassy 

1. 2g. S 

-.147 

X 




. X 

embezzle 

1 

-.131 



. • 1 . 



embitter 

1 

+ .184 




• ■ jl 

■.h.s. . 

embodiment 

1 

-.017 



• » « . 


embryology 





• * • • 

X 


emergency 

1 

+ .500 



• * . • 


• • 

emigrant 

I.S 

+ .243 




X 

• * 

emigration 

1 

+ .243 





z 

eminent domain 

l,2e, 2g,S-.819 

X 

X 


X 

X 

emissaries 

1 

-.433 

X 




X 

emoluments 






X 


emotional certainty 

«... 





X 

, , 

emotional similarity 

.... 





X 

, ^ 

empanel a jury 

S, 2g 




...» 


, , 

emperor 

1 

+ .232 

X 

X 

. « » • 


X 

empire 

1 , s 

+ .179 

X 




5 z 

employee 

1 

+ .558 



X 


. z 

employer 

l.s 

+ .558 




X 

. y 

employers association 

1 

-.192 






employment 

l.s 

+ .689 





5 X 

employment agency 

1 

+ .421 






employment bureau. 

government 

2g, S 







emporium 

1 

-1.SS5 






empower 

.... 






. X 

empress 







4 X 

emulation 

i. 2s 

-.327 






enabling act 

2g 

...... 

X 

X 


X 

« . 

enactment ^ 

encirclement policy 

S, 1 

-.063 

X 



X 

X z 

1 

-1.334 

X 

. 




enclosure 

1 

-.364 

. 




, 

encounter 

— 


• 

• 


• 

X 
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encroachment 

encyclopedia 

endogamy 

endorse 

endowment 

ends, social 

enemy 

energy, social 

enforce 

enfranchise 

engagement 

engineer 

Engel’s law 

engrossing 

engrossment 

enioin 

enlightened 

enlightened despot 

enlightenment 

enlist 

enlistment 

enmity 

enroll 

ensign 

enslave 

entail 

entanglement 

entente 

entente cordiale 

enterprise 

entertainment 

entrench 

entrepreneur 

entrust 

enumerated article 

enumerated powers 

enunciation 

environment 

envoy 

epidemic 

epidemics, social 

episcopal 

epoch 

equality 


1 

E 

ws 




o 

S 


Wesley 

Wesley Tests, College 
Editions No, , — • — 


X 

4 ) 




JB 

a 

a 


1 

-f 022 

1 

+ S90 

l.s 

+ 363 

1 

+ 084 

2s, E 

2s, E 


l.s 

+ 527 

1. 2g. S 

- 269 

1 , s' 

+ 706 

2g 


1 

- 576 

1 

+ 206 

1 

- 661 

1, 2s 

+ 048 

1 

+ 70S 

2g 

1 

+ 353 

l,2g 

+ 578 

1 

-.406 

1 

+ .416 

1 

-1 023 

1 “ 

-.776 

1 

- 966 

1, S. 2e 

+ 526 

1 

+ .716 

1 

+ 137 

2e, 1 

-1.398 

1 

+ .311 

1 

-.598 

1 

- 883 

1 

-.131 

1, 2s, S 

+ .311 

1 

-.052 

1. 2s 

+ 432 

E 


1 

-.529 

, is, S, 2g 

+ .>05 


z 

X 

X 


X 

X 

X 

X 

X 


X 

X 

z 
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College 


0 

§ 

O 

s 

& 

SA 


Mil 

II 

III 

IV 

Unpub. 

Pub. 

Kelty 

Pressey 

equalization 

2s. E 







X 

equilibration 

2s, E 







, , 

equilibration of nations 







X 

, , 

equilibrium 

2s, E 







. 

equipment 

1 

+ .70S 




■ 4 


, . 

equity 

2e, 2g, S 


X 



• • 


X 

era 

1 

+ .316 

X 



4 • 


z 

error 

1 

+ .721 







eacadrille 





■ 4 


X 

escape 

1 

+ .837 







espionage 

1 

-.628 

X 





4 X 

espouse 

1 

-.586 







establishment 

1 

+ .495 






X 

established church 

1 

-.581 

X 

X 





estate 

1 

+ .442 





X 

X 

esteem 

1 

+ .326 




4 4 

s 


esthetic 

US 

-.718 




4 • 

# 4 

, , 

estimation 

1 

+ .326 




4 4 

. . jr.h.s. . 

estrangement 

2s, £ 






, * 


ethical relativity 

• • • 4 





4 4 

X 


ethical standards 

1 

-.824 




4 • 

^ , 


ethics 

1 

-.572 




4 4 

X 


ethnology 

1 

-1.262 





X 


etiquette 

1 

+ .258 




X 

X 


euMnics 

1. 2s 

-.585 




X X 

X 


eulogize 

1 

-.333 







European 

.... 






X 


evacyiate 

1 

+ .132 

X 





zz 

evaluation of property 

2e 








evening school 

1 

+ .590 







event 

4 « 4 ■ 







z 

eviction 

1 

-.106 







evidence 

1. 2g 

+ .390 

X 

X 


X 


X 

evils 

1 

+ .747 







evolution 

1 

+ .130 




4 ■ 

X 


exaltation 

1 

+ .084 




4 • 



examination 

1 

+ .689 




4 4 


X 

exarchate 

1 

-1.623 

X 






ex cathedra 

I 

-1.402 







excesses ^ 

1 

+ .195 







eXeesaive capital outlay 

• • • • 






X 


exdiange 

i,s 

+ .526 


• 




X 

exchange, foreign 

2e 







4 4 

exchange, produce 


2e 







■ 
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Word 


i 

E 

X 

V3 


•c 

B 

O 

o 


Wesley 

^ Wesley Tests, College 
.2 Editions No. . — ■ — . 


4 ; 


^ o. ji ^ 

= = 5 £; 5 ,£ 3 


exchange, stock 
exchequer 

2e 

1 

-.660 

• 


excise 

1, 2g. S 

- 629 

X 

X 

exclusion 

exclusive 

h 

1 

+ .505 



excommunicate 

1 

- 475 



execute 

S 




executive 

1, 2g. S 

+ .300 

X 


executive act 





executor 

i 

- isi 

X 


exempt 

s 




exemption 

i 

+ 264 



exhibition of emotion 




exile 

i 

+ .632 

X 


ex officio 

l, 2g 

-.586 

X 

X 

exogamy 

• • • • 




exorbitant 

1 

+ .063 



expansion 

l,2s,E,S 

+ .390 

X 


expatriate 

S 


X 

X 

expedient 

1 

+ .005 



expedition 

1 

+ .616 

X 

X 

expel 

1 

+ .626 



expenditures 

l.S 

+ .689 


. 

experimental research 

.... 

+ .558 



experiments 

1 



expert 

1 

+ .595 



exploit (v.) 

1 

-.163 

X 


exploitation 

1, 2s, E 

-.232 

X 

X 

exploitation of labor 





exploration 

i 

+ .689 

X 

. 

explosive 

. . , . 

+ .458 


. 

exportation 

l.S 



export duty 

1 

+ .064 


. 

exposition 

1 

+ .263 


. 

ex post facto 

2g 

+ .453 

X 


exposure 

1 


- 

express riders 

1 

-.359 


- 

extension course 

l.S 

-.085 


. 

extent of market 

1 

-.824 



exterminate 

1 

+ .326 


- 

external taxation 

1 

-.919 

X 

. 

extinction 

1 

+ .263 


. 

extortion 

1 

+ .126 



extra-constitutional 

2k 



• 


jr 


X 

X 

X 

e 

X 

z 

z 


X 

« 

z 


y 

X 

z 


s. 
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e 











c 

o 





M 

t 

o 


X ^ 

^ tn 

O' .a — 2 

s a V Si 


"2 

s 


^ M 

p a. &, 

extradition 

1. 2g. S 

-.566 

X X 


X . X 

extraction 

1 

+ .258 



• p » • 

extra-legal^ 

1. 2g 

-.840 

X 


p p p p 

extra-territoriality 

1 , s 

-1.056 



p X . X 

extravagance 

1 

+ .632 



. p . p 

extremists 

1 

+ .247 



X 

extricate 

extrovert 

1 

+ .079 



X 

eye-witness 

rabian 

i 

1 

+ .632 
-1.276 

X 


X . , . 

face-value 

1 

-.195 




facilitate 

facilities 

1 

-.052 



. ! , X 

faction 

i 

+ .190 

X 


X 

factor 





X 

factory 

s 




. . . y 

factory system 

1, 2e 

-.254 



p p p p 

faculty 

1 

+ .426 



p * » p 

fallacious 

1 

-.279 



p p p I 

family 

S 




p « p p 

family expenditures 

. . » * 




X . . . 

family pattern 
fanaticism 

i ' 

-.137 



X . . . 

Far East 

1 

-.121 

X 



farm 

1 

+ .837 

• • 



farm bureau 

i,s 

-.177 

• • 



farmer 





X X 

farm loan 

s 





farm produce 

1 

+ .620 



. . 4 . 

farmstead 

1 

+ .125 



. . jr.h.s. . 

Facists 

1 

-.555 

X 


X 

fashion 

2s 





fastness 

1 

-.110 



X 

fatalism 

favorable balance of 

1 

-.691 



p p p p 

trade 

1 

-.635 



p p p • 

favored nation treaty 
favoritism 

i‘" 

+ .316 



p p p • 

fealty 

1 

-.813 

X 


p p p < 

fecundity 

1 

-.877 



* n • * 

federal ' 

1 . s 

-.028 

X 


. . z 

Federals 





. . zx 

Meral aid 
federal courts 

i 

-.133 

X 



2g 




p p p ' 
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jB 

S- ^ ^ 
s £ 'S 

1 

a. 

federal government 

1, 2g 

+ .304 

X 







federal judge 








X 


Federal Reserve Bank 

2e 







X 


federal reserve system 

1 

-.317 








federal union 

1 

-.202 

X 







federalism 

2g, S 







X 


federation 

1, 2g 

+ .042 

X 






X 

fee 

l,2g,2e,S 

+ .526 

% 

X 

X 




X 

fceble-mindcdness 

1 

+ .320 








felony 

1, 2g, S 

-.254 

K 

X 

X 



X 

X 

feminist 









X 

feminist movement 








X 


fence viewer 

2g 









fermentation, social 

2s. E 









fertility 

1 

+ .374 






. . 6 

X 

fertilizer 

1 

+ .501 








fetter 

1 

+ .263 








feud 

1 

+ .311 







X 

feudalism 

1 

-.507 

X 






X 

feudal lord 

i.s 

-.523 

X 







feudal tenure 

1 

-.655 

X 







fiasco 

1 

-.612 








fiat money 

«... 








X 


fictitious money 

.... 







X 


fidelity 

1 

+ .253 







. 

fiduciary 

.... 








X 

fief 

1 

-.628 

X 

X 





• 

field marshall 


, . V . . . 







X 

fighting machine 

i 

+ .199 

X 






• 

figure head 

1 

+ .384 







- 

filibustering 

1, 2g, S 

-.645 

X 

X 




X 

X 

filing system 

1 

-.035 






. 

- 

finance 

1,S 

+ .437 






X 

z 

finance company 








X 

• 

financier 

i, 2e 

+ .437 




X 


X 

* 

financing 








X 

■ 

fine 

2e, 2g, S 








X 

fire arms 

1 

+ .705 







♦ 

fire protection 

2g 








• 

firm 

, , 









first-class power 

1 

-.451 

X 






X 

fiscal 

1 

-.535 

X 







fisheries 

.... 


. 






X 

fixation 

2s. E 


' 

* 



— 1 
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S' 

C ■§ T3 K 
su ^ (X. 

M taN 

^ 

fixed capita! 


. 




X . . . 

flank 






X 

flag 


• . . 




X 

flagihip 


. 




x 

fleet 






t 

flood 






X 

floor 






x . . . 

floor leader 

2g. S 


X 




flotilla 






x 

fluctuate 

1 

+ 016 





fluid capital 

1 

-1 072 





focut of attention 

2s. E 






foe 






z 

folk-moot 

1 

-I 423 

X 




folkways 

1, 2s. E 

- 919 





follower 

1 

+ 637 





foment 

1 

- 302 





food 






X , 

foodstuff's 

1 

+ 484 




, . jr.h.s. , 

forage 






X 

foray 

1 

- 428 

X 




forecasting of demand 






X 

force 

S 






forced labor 

1 

- 069 





forced loans 

1 

-.480 





forces (military) 






. . . y 

forces (social) 

2s, E 






foreclosure 

I 

- 259 



X 


forejgn 

l.S 

+ 674 

X 



z 

foreign affairs 

1 

+ 374 

X 




foreign born 

1 

+ 674 

X 




foreign exchange 

1 

- 254 

X 




foreign policy 

1.2g 

000 

X 




foreign relations 

2k 

, , 





foreign soldier 






X 

foreigners 

l.S 

+ 616 

X 


X X 


foreman of the jury 

1. 2g 

-1- 616 

X 

X X 


• • « • 

forerunner 

1 

-I-.326 




• • • • 

foresight 

. • • 





X 

forester 

S 

* • . . 




X 

forgery v 

1. 2g 

+ .369 




z 

formalism 

is 





• • * * 

formal organization 






X 

fortifications 

l.S 

+ .437 




z 
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K/i 

m 

e 

c 

** 

w 

O 

2 


I 

II 

1134 

III 

IV 

Unpub. 

Pub. 

Kelty 

s 

£ 

fortress 

fortuity of human 






. 

X 

development 





X 


forum 

1 

-.064 



t • ■ 


forty-niners 





• « • 

X 

foster 

i 

+ .143 





foster home 

1 

-.3 0 





foundation 

1 

+ .574 





franc 






X 

franchise 

1.2g 

1 

b 

X 


X 

X 

franchise privilege 

2g 



. * » » » 


• 

franking privilege 

.... 


X 

X . , . . 

X 


fraternity 

1 

+ .479 



• • • 


fraternize 

1 

-.068 





fratricide 





X 


fraud 

i 

+ .448 




a 

fraudulent 





X 


free-bootcrs 

i 

-.339 

X 


. 


free competition 

1 

-.390 





free-holders 

1 

-.829 

X 


• » • 


free-thinker 

1 

-.654 




X 

free-trade 

1 

-.269 




X 

freedom 

1.2s,2g,S+.437 

X 

X , , , , 

. . 4 

z 

freedom of contract 

1 

-.972 



X 


freedom of the press 

1. 2g 

-.012 

X 


» s s 


freedom of religion 

1.2g 

+ .127 

X 


, , , 


freedom of speech 

1.2g 

+ .058 

X 


• • • 


free government 

1 

-.133 

X 


• * t 


freeholders 

. . . 




t ■ ■ 

X 

free land 

1 

+ .06i 

* 


• • • 


freedmen 

1 

-.187 

X 


t s • 


freemen 

1 

-.080 

X 


■ B • 

X 

free peasants 

French Chamber of 

1 

-.496 

X 


. 


Deputies 

ft*. 


• 


X 


friars 

1 

-.607 

. 


t « * 


friction 

1 

+ .316 



• B • 


frigate 

« t • . 


. 


. 

X 

frontier 

1 

+ .442 

X 



B B B 

z 

frontiersman 

1 

+ .442 

X 

X . . . . 

B 


fugirive 

1 

+ .616 

X 

X J 

t . 

X 

function 

1,S 

-.137 

X 


B 

. 

function, social 

E 



■ 

. 

. 

• ' 
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c/3 

M 

o 

z 

■w 


X 

B > 

.o 

S' -o 

p. pp 

■5 ii 
U a. 

functional clutter 
pattern 






» • 

X 


functional integration 






• p 

X 


functionalization 






• • 

X 


functional middleman 






» a 

X 


functional pattern 


.... 




• 

X 


functionary 

1 

-.517 




» a 



fundamental 





• • 

a • 

X 


fundamentalitt 

i 

-.^7 



» » 

• » 


X 

fundamental law 

i> 2g 

-.972 

X 


a a 

• ■ 



funding 

1 

-.783 




• p 

. X 


fundi 

1 

4-.49S 





. ir.h.s. X 

fur trade 

• • 





» k 


4 

fulion 

1, E, 2s 

-.069 







fuaion of cultures 

2s 





a 



futility 

1 

+ .053 




• p 



futures, dealing in 

2e 





• p 

X 


gage 

1 

+ .147 




• p 



gag law 

1 

-.761 

X 

X 

X 

X 



gag resolution 

« • • • 





• p 


X 

gainful occupation 

1 

+ .247 




a p 



gallantry 

1 

+ .126 




• p 



galleon 

1 

-.859 




p p 



gallety 

1 

+ .49S 




• • 



gallows 

1 

+ .658 

X 



> a 


6 . 

gambling 

1 

+ .779 




a a 



gangster 

1 

+ .589 





X 

X 

gaanson 

1 

+ .432 

X 



X 


X 

gateway 

1 

+ .553 




a P 



gauntlet 

1 

-.063 




a a 



gendarme 

• < ■ • 





a • 


X 

geneological continuity 

. . . • 





» a 

X 


geneological history 

• . > . 






X 


general 








z* 

generalissimo 

1 

-1.308 

X 

X 





generalized concepts 

.... 






X 


general search warrant 

1 

-.725 

X 



p p 



general staff 

1. 2g 

-.306 




a a 



general store 

1 

+ .432 




• a 



general strikes v 

1 

-.317 







general wdfare 

1 

+.069 

X 



X X 



generations 

i.s 

+ .253 




X 

X 

X 

genesis, social 

2s, E 





► 



genetic psychology 






• 

X 
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I 

O 

s 

& 

1 

Mil 

11 

I 



.A 

S. 

e 

O 

.o 

n 

ft. 

1 1 

genetio (mucturet) 

1.2s 

-1 018 






X 


gentuf 

Gentile 

1 

+ 057 

- 





• 

X 

gentlemen’s agreement 

1.2g 

- 576 

X 





. 


gentry 

1 

- 603 







X 

geography 









X 

gerrymandering 

1. 2g, S 

- 866 

X 





X 

X 

gift 

1 

- 705 








gilt-edged 








X 

t • 

ghetto 

i 

- 633 







X 

gladiator 

1 

+ 174 








0obe 









X 

gold bullion 

2e 









gold certificate 

2e 




X 

X 


X 


gold coast 

1 

- 687 








gold, international flow 
of 

2e 









gold reserve 

1 

-.698 








gold standard 

1, s 

- 692 






X 

X 

good will 

1 

-.076 







* « 

goods 

i.s 

-f- 431 






X 

z 

gospel 

1 

+ 300 








govern 

s 

+ 558 







X Z 

government 

l,2B,2g,S 

X 


X 

X 


X 


governmental functions 

P « . . 

-.116 






X 

• • 

government interference 1 

X 





. 

r t 

government operation 

1 

- 070 

X 






r » 

government ownership 

1 

-I-.20S 

X 





. 

X 

government services 

i.' 2g, s 

+ 616 

. 





X 


governor 

X 





. 

. z 

governor general 

1 

+ .553 

X 





X 

* • 

gradation 

2b, E 

.... 






. 


graduated 

«... 

+ .553 

. 





X 


graft 

I.S 

X 






z z 

grandchildren 








X 


grandee 

. . • . 








X 

grandfather clause 


-.127 

X 

X X 

X 

X 


X 

* • 

grand jury 

i 

X 





> 

X 

granger 

1 

— .434 







X 

grant 

1. 2g. S 

+ .505 

X 





X 

zz 

gratification 

2s 







. 

• 1 

neat charter 
‘great masters” 

S 

-.444 

X 





X 

• ^ 

1 

- 





• 

* m 

“great powers” 

1 

-.244 

X 





• 

2 
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1 

o 

s 

41 

■g 


^ > 
^ 

^ 

3 

^ mO 

D cu 

>> 

w m 

■v t 

biC On, 

great leal 

2g 







greed 

1 

-h S53 




X 


greenback 

1, 2e 

- 144 



X 

X 

XX 

mgariouineu 
Greiham'i Law 

1, 2s 

-1.018 




X 


2e 




X 



grievance! 

1 

+ .264 






group 

2s, b, S 







group conaciouaneK 

2s 







group conflict 

1 

- 672 

X 





guarantee 

1 

+ .374 






guard 


. . . 





X 

guardian 

1 

-.553 

X 


X 


2 X 

gubernatorial campaign 2g 


X 



X 

, , 

guerrilla warfare 

1 

-.276 

X 

X 



X 

guidance 

1 

+ .405 



X X 



guide book 

1 

+ .437 






guildf 

1 

- 334 






guild iocialiam 

1 

-1.218 






guillotine 

• • « • 






X 

guiltv 







X 

gun boat 

«... 






X 

gun powder 
habeas corpus 

l.s 

- 714 

X 

X 

X X . X 


4 . 

X 

habitation 

1 

+ .132 






habits 

s 







habitual criminal 

1 

+ .058 

X 





hamlet 

1 

+ .132 






hand-bill 

1 

+ .463 






Imndicraft 

1 

+ .005 





X 

hangar 







X 

hanger-on 

i 

+ .079 

X 





harangue 

1 

-.200 

X 

X 

X X X X 



harass 







X 

harbor 

i 

+ .658 





X 

hardships 

I 

+ .716 






hard times 

1 

+ .558 





X 

harem 

1 

+ .063 






harmony 

1 

+ .384 






harrow 


f . • 





5 . 

harry (v.) 

i 

-.616 

* 


, , , , , 

• • « 

, , 

harvest ' 







X 

haul (long, short) 

s 



• 



• 

head money 

1 

-.846 






head of family 

1 

+ .437 





4 . 
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headquartert 

1 

+ 674 


head tax 

1 

- 328 

X 

health 

1 

+ 690 


hearing 

2g 



heathen 

heckling 

1 

-f- 357 

X 

hedging 

heir-apparent 

hemisphere 

1 

- 572 

X 

hemp 

henchmen 

1 

-t- 189 


hereditary 

1 

-f 205 


hereditary nobility 
hereditary similarity 

1 

-.139 

X 

hereditary traits 

^ , 



heresy 

1 

- 218 


heretic 

1 

- 177 


heritage 

1 

+ no 


heritage, social 

2s, E 



hermit 

1 

+ 379 


hero 

1 

-I-.616 


heterogeneity 

heterogeneous 

i " 

-.756 


hetman 

1 

-1 696 


hierarchy 

1 

- 976 

X 

high cost of living 
highness 

1 

+ 120 


high seas 

i 

.000 


highways 

s 

. . 


Hindu 

* « * 

. 


hinterland 

1 

- 704 


hog-reeve 
holding company 

2g 

1, 2e 

-.935 


holdings 

1 

-.202 


homage 

1 

-.186 

X 

home 

S 

. » . . 


home-rule 

1. S 

-.269 

X 

homemade 

1 

-.800 


homestead 

1.S 

-1-.2S8 


homicide 

1 

-H.064 

X 

homogeneous 

1, 2s 

-.545 


honorariums 





honors 

i' 

+ M7 


horde 

1 

+ .153 
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K/) 

m 

c 

i 

o 

S 

«> 

-3? 

o 


M ^ M 0^ 

^ ^ 

Unpub. 

Pub. 

X a 

*-■ w 

horizontal combination! .... 





X 


hoipitable 

1 

-I-.458 





6 . 

hospital 

1 

-I-.837 






host 

1 

-1-.584 





X 

hostile 

1 

+ .379 





z 

hostile attitude 

• • t • 





X 


hotel 

1 

+ .942 






hours, maximum 

2e 







House 







X 

House of Commons 

i 

+ .124 

X 




X 

House of Lords 

House of 

1 

+ .057 

X 




X 

Representatives 

2g, S 






z 

household industries 

1 

-.096 






housing conditions 

i.s 

-.059 



X 



hovel 

1 

+ .158 






humane 

1 

+ .206 



X X 


6 . 

humanism 

1 

-.892 






humanitarianism 

1 

-.329 






humanity 

1. 2s 

+ .262 






human nature 

2!, E, S 







humiliation 

1 

+ .389 






hundred 

1 

+ .139 






hunger strike 

1 

-.084 






hustings 

.... 





X 


hut 

1 

+ .633 






Iprbrid 

1 

-.426 






hygiene 

1 

+ .267 



X X 


X 

hypocrite 

1 

+ .182 






hypothesis 

1 

-.362 






hypothetical case 

. • • • 

-.921 




X 


iconoclast 

1 






idea-forces 

2s 







idealist 






X 


idealization 

is, E 







ideals 

1. 28. S 

+ .395 






ideas 

1 

+ .512 





5 . 

identification 

2s,E 







identity of interests 






X 


idiot 

1 

+ .512 



. , X 


6 . 

idler 

1 

+ .512 






idol 

1 

+ .456 






idolatrous 

1 

+ .312 






ignominy 

1 

-.349 

• 

> 

. 
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E 
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S 

c 

S 

o 

s 
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- ^ > 



illegal 

illicit traffic 
illiteracy 
illiterates 
illusion 
imbecile 
imitation 
immemorial 
immersion 
immigrant 
immigration 
immigration 
immorality 
immortality 
immunity 
impair 
impanel 
impartiality 
impassable 
impeachment 
imperial 
imperialism 
imperil 
impersonality 
implied powers 
import 
import duty 
importation 
impose 
impost duty 
impoverishment 
impregnable 
impression 
impressment 
imprison 
improvement 
impunity 
inalienable 
inaccessible 
inauguration 
inbr^ing 
incentive 
incite 
inclosure 


restricted 


1. s 

+ 456 

1 

- 224 

1. 2g, S 

+ 249 

S 


2s 


1 

+ 278 

1, 2s, E 

+ 673 

1 

+ 006 

S 


1, 2g, 2e, S+ 456 

It 


1 

+ 389 

1 

+ 327 

1. 2g 

- 127 

1 

+ 1S3 

1 

+ 395 

1, 2g. S 

- 110 

1 

- 005 

1.2g,2s,S~ 715 

1 

- 442 

1 

- 444 

1 

- 708 

1 

+ 406 

2g, 2e 


1 

+ 395 

1 

+ 453 

1 

- 555 

1 

+ 022 

1 

- 170 

1 

+ 458 

1 

- 475 

1 

+ 574 

1, s 

+ 632 

1 

- 287 

1 

+ 395 

1, s 

+ 522 

i 

+ 337 

1 

+ 09S 

1 

+ 027 


X 

X X 


X 

X 


X 

X 

X 


X 

X 


X 

X 

X 

X 


XXX 
X 




X 

z 

zz 


X 

z 
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Word 


3 

t 

.e 

i 


Wesley Tests, 

Wesley 

Gillege 

M 

Editions No. 

X 

Vi 

« 

1 

O 

1 


M 

S 

i- -i 


s 


^ 

^ ^ 

;= c£ 


inclusive 

1 

+ .285 

income 

1. s 

+ .558 

income, net 

2e 


income tax 

1, 2e. 2g 

+ .201 

incompetence 

I 

+ .269 

incontrovertability 



incorporation 

1.2g 

-.175 

incrimination 



incumbencies 



incumbent 

1 

-.228 

incursion 



indebtedness 

1 

+ .401 

indemnity 

1.2e. 2e.S-.18S 

indenture 

1 

-.809 

independence 

1 . s 

+ .322 

independent 



indeterminate sentence 

if 2 g, s 

-.693 

index of prosperity 

1 

-1.224 

index number 

1. 2c 

-.866 

indictment 

1. 2g, S 

-.391 

indiiferentism 

2s 


indirect tax 

1. 2g 

-.466 

individual 

28, S. E 



individual initiative 

individualism 

individualists 

iodorse 

indorsement 

indorser 

indulgence 

industrial 

industrial accidents 
industrial center 
industrial conflict 
industrial democracy 
industrial depression 
industrial interests 
industrialism 
industrial revolution 
industrial "satellites’’ 
industrial union 
Industrial Workers of 
tlw World 


1 , 2s, 2g, 2e-.798 

1 

+ .148 

1 

+ .079 

1 

+ .016 

1 

+ .195 

i.s 

+ .390 

i " 

+ .195 

1 

-.324 

l. 2e 

-1.077 

1 

-.272 

1 

-.266 

1, 2s 

-.550 

1,S 

-.319 

1 

-1.287 

1 

-.707 


X 

X 

X 


X 

X X 


X 

X 

X 


a 

X 

z 


2e 
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s 

jC 

e 

e 

1 

z 

Hk 

X 

Wesley Tests, 
Editions No. 

M M ^ 

M ^ M H>4 

Wesley 

College 

-4 -I 1 
B £ 3 1 

industry 

1, 2e, S 

+ m 




■ 

industry, infant 

1.2e 

- 403 



X X 


industry, key 

2c 






industry, localization of 2e 






inefficient 

1 

+ .433 





inelastic currency 

1 

-1 098 





inequality 

1 

+ .390 

X 



. jr.h.s. . 

inequality of distri- 
bution 

2e 





inequality, social 

25 






inevitableness of 
socialism 






X . . . 

infallibility 

i 

-.376 





infant mortality 

1 

-.121 





infantry 

« « « ♦ 





X 

inferiority 

1 

+ 390 




• • • » 

infidel 

1 

-.011 





infiltration 

2s, £ 






inflation 

1 

- 698 




X X 

inflict 

1 

+ 374 

X 




influence (v.) 

1. 2g 

+ S33 

X 



X X 

influx 

I 

+ .022 




• • • • 

informant 

I 

+ 195 




• • • 1 

infraction 

I 

- 106 




• * « • 

infringement 

I 

+ 037 

X 

• N « 


X 

in-group 

E 



• ♦ • 


• < ■ ■ 

inhabitants 






z 

inherent rights 

1 

-.750 


X 

, 

• » • • 

inheritance 

1, 2s 

+ .501 





inheritance laws 






X 

inheritance tax 

i. 2g 

- 079 

X 

p . • 


X 

iniquity 

1 

+ 085 


. 


• a « e 

initiative 

1, 2g, S 

+ 079 

X 

X X X 

X X 

X X X 

injunction 

l,2g,2e,S_.‘i76 

X 

XXX 

X 

X z 

injustice 

1, 2s 

+ 437 

X 



> • ■ X 

inland waterways 

1 

- 005 





inmate 

1 

+ .085 



X X 

X 

inn 

1 

+ .532 





innate tendency 

1 

-.903 





innocuous 

1 

— .929 





innovation 

1 

+ .116 





inoculate 

1 

-.052 





inquest 


• • • - 

X 

X 



inquiries 


. . . 


. * 

• * 

a X • • 
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inquisition 

inroads 

insane asylum 

insanity 

inscription 

insolvency 

insolvent 

inspection 

inspector 

inspiration 

instability 

instability of labor 

instability, social 

installation 

instalment 

instalment buying 

instinct 

institute 

institution 

institutional care 

instructions 

instrument 

instrument of govern- 
ment 

insubordination 


t/} 

M 

‘J3 

Morton 

Wedey 



X 


IV 

Unpub. 

1 

-.660 

X 

X 

X 

X 

X 

X 

1 

+ .137 







1 

+ .664 







1 

+ .501 







1 

+ .253 







2e 







X 

X 

s 

+ .426 

X 






s 








l.s 

+ .258 







1 

+ .022 

X 






2s" 








.... 

+ .316 







.... 

-.207 







1 

+ .137 







1, 2s, E, 

S-.005 

X 








X 






i 

+ .505 

X 







insurrection 
intangible 
intangible property 
integrate 
integration 
intemty 
intraectual 
‘intellectual horizon 


"insular cases" 

2g 












insular 











X 

insurance 

i 

+ .369 










X 

insurance companies 










X 



insurance, liability 

2e 






X 

X 





insurance, life 

2e 






X 

X 





insurance, marine 
insurance policies 
"insure domestic 

2e 






X 

X 





.... 






■ 

• 


X 



tranquility” 

2g 












insurgents 

2g 

...... 










X 


253 X 
115 . 
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Word 


intellectualism 
intelligence 
intelligence quotient 
intelligentsia 
intemperance 
intensive farming 
intention 
interaction 
intercommunication 
intercourse 
interdependent 
interdict 
interest 
interest rates 
interests 
interference 
interior 
interlocking 
directorates 
intermarry 
intermediary 
intermittent war 
internal 
internal affairs 
internal improvements 
internal taxation 
interna] waterways 
international 
international adminis- 
trative union 
international 
arbitration 
international 
commission 
international congress 
international 
cooperation 
international court 
international exchange 
international law 
international peace 
international relations 
international trade 
internationalism 






Wesley 

3 

E 

U 


Wesley Tests, 

College 



Editions No, 

— ' — ■ 

X 

Ui 

m 

c 

o 

* 1 * 

c 

>s 

u 

3 

X - . 



s 

s 

I 

II 

11 

II 

rv 

b S 




1 

-1.062 

1 

-I-.185 


i.s 

- 058 

X 

1 

-.132 

X 

1 

-.311 

X 

1 

+ .137 


1 , s 

+ .311 

X 

2g 


• 

1 

-.154 

X 

1 

-.455 

X 

1 

-.137 

X 

1 

-.155 

X 

1 

-.079 

X 

ii 2g 

-.264 


1 

+ .069 

X 

% 

+ .127 

X 

1 

+ .127 


2s 




X 

X 

X 



X 

X 

X 

z 

4 

Z 
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0 

W 
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College 


1 

s 

C 

§ 

O 

s 

X 

•a 

% 


^ M mm ^ 

Unpub. 

Pub. 

Kelty 

Pressey 

interoceanic 

1 

-.459 

, 




, 

interpellation 

1 

-1.125 




X 

. 

interpenetration 

2*, E 







. 

interpretation 

1 

+ .016 

X 


X X 


X 

interregnum 

1 

-.882 

X 

X 

X . X . 



interstate 

S 

+ .121 

. 




X 

interstate commerce 
Interstate Commerce 

1 





■ 

Commission 

1. 2e. S 

— .466 

X 





interstate trade 

2g 




X X 

X 

, , 

interstimulation 

2s, E 

...... 






interurban 

1 

-.065 






intervention 

1 

-.052 

X 

X 

X X X X 


X 

intestate 

1 

-.745 





, , 

intimidation 

1 

+ .258 






intolerance 

1 

+ .322 




• • 

Y 

intrasute 

i.s 

-1,176 

X 

X 

XXX. 

« • 

X 

intrastate trade 

2g 

...... 




• • 

, , 

intrigue 

1 

+ .264 

X 




X 

intrinsic value 






X 


introduction 

1 

+ .632 





5 . 

introspection 

, . . . 





X 

. 

introvert 

.... 

... . . 




X 

, , 

intruder 

1 

+ .411 




• j 

r.h.s. . 

invalid 

1 

+ .637 





zz 

invalidate 

.... 





X 


invasion 

1 

+ .553 






invention 

1, 2s, S 

+ .616 





z 

inventory 

2e 

+ .632 



X X 


. « 

investigation 

1.2g 

X 

X 


X 

X X 

investment 

1 , s 

+ .352 




X 

X 

investor 

1 

+ .522 




X 


investiture 

1 

-.883 

X 





invincibility 

1 

-.042 

X 



X 

X 

inviolable area 

• • • • 





X 


inviolability 

1 

-.537 

X 



X 


invisible government 

2g 







ipso focto 

1 

-.772 





, , 

irony 

1 

+ .009 





, , 

irreconcilable > 

. • • • 







irrigation 

I.S 

+ .574 




X 

4 z 

Islam 

4 • ■ • 





X 


island 

• • » ■ 







X 

isolate 

1 

+ .137 




■ 

• 
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isolation 

tuue* 

iiiuance of mone/ 
italic 

items of ttade, invisible 

items of trade, visible 

itinerant 

Jacobinism 

jail 

Japanese 

Jew 

jeweler 

jingoism 

jobber 

joint control 

joint occupation 

joint resolution 

joint session 

joint stock company 

joker 

journalism 

journeyman 

joust 

judge 

judgment 

judicial 

judicial decision 

judicial interpretation 

judicial powers 

judicial prejudice 

judicial reform 

judicial review 

judiciary 

junction 

junker 

junta 

jurisdiction 

jurisprudence 

jurist 

juror 

jury 

jury, trial (by) 
jus sanguinis 
jus soli 


+ 206 
-.20b 


1 

1 

I 

1, 2g, S 
1. 2g, S 
S 
1 
1 


2g 

1, 2g, S 
1 
1 
1 

1, 2g, S 

1. 2g 

1 


512 X t 
199 . . 

132 X . 

121 X . 

247 X . 

212 X . 


14B X 
384 , 

039 . 

138 . 

252 X 
536 X 

080 X 
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justice 

1, 2g, 2s, S-H 369 

justice of the peace 

1. 2g, S 

-1- 233 

juitice. associate 

2g 


justice preventive 

2g 


justiciable 

1 

-1 079 

justification 

1 

-1- 337 

juvenile 

i,s 

- 374 

juvenile court 
kaaba 

1, S 

+ 247 

kaiser 

1 

+ 127 

keynote 


khedive 

1 

- 827 

kidnap 

km 

1 

+ 495 

kindred 

king 

1 

+ 321 

kingdom 
king's aid 

1 

+ 547 

. 


kinship 

2s 


kinsman 

1 

+ 384 

knight 

1 

-f 242 

know-nothings 

• . . 


Koran 

1 

- 391 

labor 

l.s 

-1- 501 

labor, child 

2e 

.... 

labor, conditions of 
labor disputes, arbitra- 

2e 


"tion in 

2e 


labor federation 

1 

-.212 

labor inspectors 

. • • « 

.... 

labor problem 

1 

-1- 057 

labor rent 

1 

-1.048 

labor savings 

1 

-1-,311 

labor turnover 

2e 


labor union 

LS 

+ .258 

labor, unskilled 

2e 


laboratory 

1 

+ .448 

laborites 

1 

-.969 

lackey v 

1 

-.580 

lady 

1 

+ .558 

laissez-faire 

1. 2e 

-.838 

land 

• . • • 

« • . . » 

landed aristocrats 

1 

-.528 


Wedey 

^ Wericy Tettt, College 
.2 Ediuons No. * — * — • 
^ . 

± 3 

^ ^ 

^ M ^ M ^ Ipj 
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m 

i 
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Editions No. • *■ > 

0 t 

1 1 
s ^ 

- * = > H 1 1 

land'grant college 

1 

- 7(,9 


land-iocked 

1 

- 026 


landlord 

1 

+ 700 

, 

land, marginal 

2e 



landmark 

1 

-1- 311 

. 6 . 

land ownership 

lands 

landslide 

1 

-h 327 

5 . 

X X 

X 

land survey 

1 

+ 211 


land tax 

1 

+ 006 X 


lapse 

^8 


X 

larceny 

X 

XX X X , X 

large-scale production 
latent social pattern 
latent usefulness 
latinized 
latitude 

1 

+ 005 

X , 

X 

X 

X 

z 

law 

lawful 

1.2g, E.S 

-h 501 X 

X t 1 

X 

lawlessness 

1 

-H 474 X 

• 

law-making 
law of supply and 

S 

- 459 

* 

demand 

lawsuit 

1 

X X 

lawyer 

1 

-1- 827 X 

X X 

laxness 

1 

-f- 269 X 


layman 

1 

— 064 

X 

leadership 

1, 2s, S 

-h 664 

X 

league 

1. 2g 

+ 301 X 

z 

5 X 

League of Nations 

1, 2g, S 

-h 253 X 

lease 

1 

+ 253 

XX X 

leave to print 


X 

■ 

lecture 

1 

4- 626 

X 

“Left” 

1 

— 718 X 


legacy 

1 

- 074 


legal 

legal increment 

S 

X 

X y 

X 

legalism 

legality 

2s 

- 6% X 

- 455 X 

X 

legal personality 

1 

* * 

legal reform 

1 

' 

legal tender 
legal term 
legal writ 

1.28 

— 443 X 

X X . . 

X 

X 
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Word 

3 

e 

S 

M 

•C 



Wedey Tests, 
Editions No. 

Wesley 

College 



X 

c/a 

M 

C 

O 

w 

c 

s 

X 


MM MM 

= > 

Unpub. 

Pub. 

w 

tit! 

tt 

0. 

legate 

1 

-.717 

X 







legation 

1. 2g, S 

-.275 

X 

X 





X 

legend 

1 

+ .190 








legion 

1 

+ .116 

X 






X 

legiilation 

2g, S 

+ .458 

X 





y 

z 

legislative assembly 

1 

+ .122 

X 






zz 

legislative reference 

2g 









legislative responsibility .... 






X 



legislator 

1. 2g 

+ .532 

X 




X 



legislature 

2g.S 


X 

X 



X 


z 

legitimacy 

1 

+ .069 

X 






X 

leisure 

I, 2s, S 

+ .332 






6 

, 

leisure class 









X 

lettre de cachet 

1 

-1.497 

X 







letter of credit 
letters of marque and 

2e 


• 






• 

reprisal 



X 







levy 

l.S 

-.117 

X 






X 

liabilities 

1, 2e, S 
1, 2g. S 

+ .062 




X X 



X 

libel 

-.408 

X 




X 


X 

libelous 

t • • 





X 



liberal 

i 

+ .258 

X 




X 


X 

liberalism 

S 








liberalization 

2s 

. • . . . 








liberation 

1, 2s, E 

+ .116 

X 






X 

liberty 

1. 2g, S 

+ .448 

X 

X 




X 

z 

liberum veto 

I 

-1.581 

X 







license 

l,2g, 2c,S+.2S8 

X 



X X 


z 

X 

lieutenant 

1 

+ .474 

X 






X 

lieutenant governor 

2g, S 









life organization 

£ 









tilce'mindedness 

2s, E 









limitation 

1 

+ .311 

X 

X 

X 




z 

limitation of output 

l,2e 

-.598 

. 







limited monarchy 

1 

-.418 

X 

X 






lineage 

1 

-.189 








linguistic 

1 

-.285 








liquidate 

1 

-.613 




X X 

X 


X 

literacy 

1, 2g 

+ .005 

, 




X 



literature ' 







X 



litigant 

. . . • 


, 




X 



litigation 

2g 


X 

x 



X 



litUBMIS 

litm^eatre 

.... 

-.315 

• 



. 

X 
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Word 


2 

£ 

Kn 


I 

§ 

S 

s 


Weiley Teit*, 
Edictont No. 


Wwley 

College 



livelihood 

1 

+ .079 

live stock 

living wage 

1 

- 249 

loan 

1, 2c 

+ 490 

loan, collateral 

2e 


lobby 

2g. S 


lobbying 

lobbyist 

1 

- m 

local 

local affairs 

i.s 

+ .385 

local custom 

1 

+ 310 

local government 

1 

+ 310 

local option 

2g. S 

+ 289 

lock-out 

1 

lock-step 

1 

+ 169 

locomotive 


+ SOI 

lodge 

1 

logic 

1 

- 295 

log-rolling 

1, 2e. S 

-7 03 

longitude 



lookout 

s 


loom 

. 4 . . 


loose 


-.524 

loose-construction 

1 

loot 

1 

+ .285 

lords 

1 

+ 373 

loss of citizenship 

2g 

+ 274 

lotteries 

1 

love 

1 

+ .558 

lower class 

1 

+ .264 

lower house 

I 

+ .145 

lower taxes 


- 541 

loyalist 

i.s 

loyalty 

1, 2g, 2s 

+ .384 

lucrative 

1 

— 238 

lump of labor theory 

2e 

+ .201 

lump sum 

1 

lust 

1 

+ .125 

luxuries 


-.138 

luxury tax 

i 

lynching 

I.s 

+ .379 

machine 


+ JOS 

machinery 

I.s 

machine gun 




X 

X 


X 

X 

X 

X 

X 


X 

% 


X 

X 


X 

X 
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,3 

i 
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Wesley 

College 

1 

Kelty 

Presscv 1 


MW WM 

MM 

> 
*— 4 

machine polttici 

1 

-.453 

X 









made work 









X 



magazine 

1 

+ .505 







. 

. 


magisterial district 

i . . « 








X 



magistrate 

1. 2g. S 

+ .122 

X 







X 


magnanimity 

1 

-.675 










magnate 

1 

-.385 










Magna Charta 

S 










X 

mail 

l.s 

+ .837 








4 

X 

mail order house 

1 

+ .779 







X 

X 


mainland 

1 

+ .442 









X 

maintenance 

1, 2e 

+ .379 




X 

X 





majesty 

1 

+ .352 

X 








X 

major 

.... 










X 

major domo 









X 



majority 

1. 2g, S 

+ .516 

X 

X 

X 




X 

X 

z 

majority rule 

.... 








X 


« 

maladjustment 

1, 2s, E 

-.196 




X 

X 





maldistribution 

1. 2s 

-.755 










maldistribution of labor .... 








X 



malfeasance 

« . 








X 



Malthusian 

1 

-1.413 




X 

X 





maltreat 

1 

-.032 










make work policy 

2e 











malingering 

2e 











m^mal 

.... 








X 



management 

1 

+ .448 










manager 

i.s 

+ .705 

X 








X 

mandamus 

l.s 

-1.182 

X 

X 





X 


X 

mandarin 

1 

-1.023 

X 









mandate 

1. 2g 

-.703 

X 

X 





X 


X 

mandatory 

1.2g 

-.707 

X 









maneuver 

1 

-.047 

X 








X 

manhood suffrage 

1 

+ .241 

X 









manifestation, social 

E 











manifesto 

1.2g 

-.651 

X 

X 







X 

manifest pattern 









X 



maniken ^ 









X 



manipulation 

i 

+ .011 










mannish 










X 


manor 

1 

-.407 

X 








X 

manorid court 

1 

-.908 

X 









manorial system 

1 

-.908 










man power 

I 

+ .258 

X 

■ 
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mansion 

manslaughter 

manual labor 

manufacture 

manuscript 

marauder 

march 

marginal land 

marginal price 

marginal production 

marginal utility 

marginal vendibility 

marginal wage 

margins 

margrave 

marine 

mariner 

maritime 

maritime law 

mark 

market 

market channels 

marketing 

market performer 

market price 

marque 

marquis 

marriage 

marriage rate 

marshal 

martial 

martial law 

mart 

martyr 

marxist 
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massacre 
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mass production 
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materialism 
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1 
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1 

+ .374 

X 

1 
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+ .236 

X 
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X 

1 
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1 
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1 
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1 

+ .416 

■ 

1 ' 

+ .327 

X 

1 

-.312 

. 

1 

+ .495 

. 

I 

+ .442 
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-.613 

• 


a 

X 
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material well-being 
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matrimony 

mature 

maturity 

maximum 
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melw 
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menace 
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mercantilism 
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merger . * 
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merit . 


I 

-.403 

1 

4-. 264 

1 

-1.366 

1 

+ .322 

.... 

+ .253 

1 

+ .384 

1. 2g 

+ .479 

1 

+ .322 

i.s 

+ .443 

i.s 

+ .500 
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• ••••• 

1 

+ .0005 

I.2g 

-.122 

i 

+ .016 

I 

-.403 

1 
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i 

-.382 

s 


i 

+ .199 

1 

-.079 

1 

-1.175 

I 

-.754 

1 

-1.387 

i 

-.149 

I, 2e 

-.866 

1 

-.481 

.... 
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-111 

1 

+ .169 

2e 


.... 
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X 


X 


X 


X 


X 


X 


X 


X 


X 


X 
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X 
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1 
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1 
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Metternichism 

1 

-1 204 
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1 
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middle-class 

1 
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middle-man 

1 

- 118 


X 

migration 

1, 2s 

+ 17i 



militarism 

1. 2g, S 
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military 
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1 
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1 
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military discipline 

1 

+ 2(A 
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military league 

1 
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military posts 

1 

+ 074 
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military power 




X 

military regime 




X 
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1 
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X 
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minimum 

1 
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minimum wage 
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X 
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X 
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X 
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A 
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z 
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1 
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mobilization 

1 
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z 

mode 

1 
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model 

1 

+ 352 





5 


moderation 

1 

+ 426 





X 


moderator 



X 

X 





modern 

S 







X 

modern family 






X 



modernist 

1 

- 577 
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+ 315 
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Mohammedanism 

1 
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X 



mollify 

1 

- 217 







monarch 

ES 

+ 448 

X 

X 




X 

monarchist 



X 

X 

XXX 




monarchy 

1. 2g. S 

+ 448 

X 
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z 
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1 

+ 321 





z 

z 
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1 
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monetary 

1 
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X 
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2e 




X X 
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money economy 
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. 
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x 
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1 
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1 
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Monroe Doctrine 
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X 
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moral standard 
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* 

X 

1 
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1 

- 665 

. • • 

s 
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s 

X 

X 

1 

+ 448 
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l,2g 

- 597 X 

X 

1, 2g 

-1 140 X 

X 

2s 



l.S 

- 189 X 

X 



X 

1 

- 100 X 


2g 


X 

1 

- 005 X 



1 


- 919 

- 369 X X 


mutual exchange 
mutually favorable terms 

4- 080 
- 919 

mythology 

nabob 

1, 2s 

1 

nation 

2g, 2s, S 


national 

national assembly 

1 

- 207 

national bank 

1, 2e 

4" 126 

national bank notes 

2e 

- 528 

national church 

1 

national debt 

1 

4- 158 

national defense 

1 

4- 269 
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national domain 

1. 2g 

national government 
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X 

X 
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X 

X 

a 


n 

X 

X 
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nationalism 

1, 2s 

-.339 
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nationality 







nationalization 

1. 2s 

- 517 

X 




national parks 

2g 






nationals 

2g 


X 




national suffrage 






X 

native-born 

1 

-b 322 





native soldier 






X 

natural boundaries 

1 

- 075 

X 




natural desires 

2s 






natural evolution 






X 

natural interests 






X 

natural resources 

1 

-b.327 





naturalism 






X 

naturalization 

1. 2g. S 

-b.l27 

X 


X 


naval 




, , 



naval stores 

I 

- 194 

X 




naval supremacy 

1 

•f 069 

X 




navigation 

i.s 

+ 327 



. • . . 


navy 







Near East 

1 

- 470 

X 


. 


necessaries 






X 

"necessary and proper’ 







clause 

2g 




* • . • 

. 

necessity of life 

1 

+ 073 



‘ • • 

, 

needs 

S 




* . . 

X 

negligence 

2g, S 




‘ • 


negotiate 

l,2g 

+ 258 

X 


.... 

X 

"negro 





• . . 


ne^ro suffrage 






X 

neighborhood 

S 





. 

nepotism 

1.2g 

-1.287 

X 

X X 



net earnings 

1 

-.630 





neutral country 


... . 




X 

neutrality 

l,'2g 

+ .174 

X 



. 

neutralize 

1 

-.253 





neutrals 

2g 




• • . « 


neutral zones 






X 

newspaper 

.... 

. . • 



. * • . 


nihilism 

1 

-1.051 
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nihilists ' 

. 





X 

nobles 

1 

+ .325 
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nobilirv 
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+ .251 
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z 
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zz 
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nominate 

nomination 

nominee 

non-competent 

nonconformist 

non-enforcible 

non-importation 

non-intercourse 

non-intercourse (act) 

nonjunng 

nonpartisan 

non-sectarian 

non-social 

nordic 

normal school 
normalcy 

Northwest Ordinance 

notables 

notary 

notary’s seal 

note 

note, Federal Reserve 

nouveaux riches 

novelty goods 

novice 

nuisance 

null 

null and void 
nullify 
nun 
oath 

obedience 

obey 

obiter dictum 

object, social 

objective 

objector 

obligations 

obnoxious 

obscene 

observatory 

observer 

obsolescence 


1 

- 491 


s 

I. 2g, h 

4- 20. 

X 

2k, S 

1 

- 48=; 


1 

- 676 

X 

1 

- 296 


1 

- 244 

X 

2g 

1 

- 914 

X 

l.S 

+ 005 

X X 

1 

- 212 
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1 

- 719 


1 

+ 253 


1 

- 469 


28 

1 

- 189 

X X 

l,2g 

- 005 

X X 
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+ .437 


2e 



1 

-.559 


i 

-.334 


1, 2g 

- 442 


1 

-.048 


S 


X 

1, 2g 

+ .078 

X 

l.'2g 

+ .437 

X 

2s 



S 



2g 



E 



1 

- 004 


1 

+ .322 


1, s 

+ .385 


1 

+ 184 


l' 

+ .327 


1 

+ .442 

X 

2s 
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obstacle 1 +.574 

obstruct 1 + .401 

occupation 1, S +.716 x 

occupation, blind alley 2e 


occupation, seasonal 

odium 

offender 

offense 

offensive 

offerings 

office 

office seeker 

officers 

official 

official ban 

officialdom 

official ruling 

old age insurance 

old age pensions 

old fashioned 

Old Regime 

oligarchy 

omnipotence 

one-price system 

onset 

onslaught 

ontogeny 

open diplomacy 

‘jopen door” 

open field system 

open shop 

operating capital 

operation 

operation costs 

operatives 

opinion 

opium 

opponent 

opportunist 

opportulhity 

opposition (pvl<). 

opprenion 

optimism 

option 


2e 

> . ... 


1 

-.317 


1 , s 

+ .632 

X 

S 


X 

1 

+ .368 

. 

1 

+ 626 


1, 2g, S 

+ .779 

X 

1 

+ .668 

X 

S 


X 

1 . s 

+ .632 

X 


1 

-.455 

. 

2e 



1 

+ .569 


1 

-.966 

X 

1.2g 

-.823 

X 

1 

-.370 


1 

-.813 


1 

-.089 


1 

+ .0795 


.... 

-.723 

X 

l.s 

-1.056 

X 

1 

-1.166 


1 

-.560 


i 

+ .437 
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+ .067 


1 

+ .579 


1 

+ .327 


1.2g 

+ .626 

X 

1 

-.635 

X 

1 

+ .574 

. 
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X 

1 

+ .495 

X 

1 
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X 


X 


X 

X 


X 


X X 
X X 
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X 
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X X 


X 


X 
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X 

X 
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X 
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X 

z 
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X 
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order 

S 






X 

X 

orders (v.) 

1 

+ .500 







order of the day 


-.608 

X 






order, social 

2s, E 








ordinance 

1.2g 

-.081 

X 





X X 

ordination 

1 

-.482 







ore 








X 

organic law 

i 

-.914 

X 

X X 


X 



organism 

l.E 

+ .005 







organization 
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X 





X X 

organized labor 

1 

+ 216 







organized opposition 








X 

organizing life principle 2$, 2e, E 








origin 

1 
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original jurisdiction 

1 
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X 






originality 
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origin of state 

2s 








origins, human 
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origins of society 

. . • 







X 
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1 

+ .443 
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orthodox 

1 

-.195 






X 

ossification, social 

2s, E 







X 

ostentation 

1 

-.112 







oust 

1 

+ .090 







outbteak 








X 

outcast 

i 

+ .316 







outdoor relief 








X 

outflank 








X 

out-group 

E 








outlaw 

1 

+ .364 

X 





X 

outlawry of war 

1 

-.439 

X 






outlay 

1 

+ .143 







outlet 

.... 







X 

outpost 

1 

+ .211 






. jr.h.s. X 

output 

1 

+ .258 






X 

outrage 

1 

-.458 






, 

outvote 

1 

-.047 

X 





. . 4 

overcapitalization 
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-.825 






X 

overdraft 

1 
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. 

overhead 








X 

overhead charges 

i 
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X 
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overproduction 
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over-seen 
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over-seer 
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+ .379 
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overt 
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X 
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overwhelming 
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pacifist 

s 

.... 







X 
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panel 
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X 
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z 

papacy 
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z 

paper blockade 

1 
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X 
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+ .114 
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parade 

1 

+ .626 
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1 

+ .068 








parcel post 

28 . 




. . X . 
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1, 2g, S 
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X 
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X 
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park 
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parliament 
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parliamentary bodies 
parliamentary 
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government 

parliamentary 

1.2g 
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procedure 

1 
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X 
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1.2g 
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X 


partial 
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participation 
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partisan 
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partition 
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partnership 
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party 
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party politics 
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. 





. . 
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1 

X 

X 




X 

y 

patrician 

1 
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X 

X 






patrimony 

1 

- .750 
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patriotism 
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X 





z 

patrol 

1 
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patronage 
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. 






patroon 
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peace movement 
peaceful penetration 
peasant 
pedagogue 
peerage 
peers 

penal colony 
penal law 
penal system 
penalty 
penance 

pending .... 

penetration 1 + 

penitentiary 1, 2g, S + 

peninsula 1 + 

penology .... 

pension 1, 2g +.501 x 

pension system 2e 

peon 1 

peoples 1, S 

pep^capita .... 

perfidy 1 

periodical 1 

periods 1, S 

perjury 1, 2g 

perpetrate 1 

perpetuation 1, 2s 

persecute 1 

person £ 

personal S 

personal declaration 2e 

personal disorganization E 

personal exemptton 2e 

personality 1, 2s + . 253 

personality patterns E 

personality type E 


.050 

.385 

Y>16 

.616 


.201 

.069 

.940 

.188 

.291 

.392 

.444 

.410 

.527 

,555 

.374 

,333 

206 

,501 

,442 


.670 

.553 

'.m 

.385 

.495 

.090 

.195 

.084 

.443 


X 


X 



X X 


X 


X 


X 


X 


X 


z 


z 


X 

X 

X 

X 

X 

X 

X 


y 


z 


z 
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% 

44 

Wesley Tests, 
Editions No 

- 2- 

> 

Wesley 

College 

3 

& -5 

n. 

>s 

^ 1 

tid b 

personalization 

2s. E 









personal 

1 

- 940 

X 







personal liberty 

1 

- 370 

X 







personal property 

i.s 

- 117 








personnel 

1 

- 185 








persuasion 

1 

+ 389 








perversion 

2s, E 









pervert 

1 

- 338 








petition 

1. 2r, S 

+ 506 

X 






X x 

retition of Right 

S 









petit jury 

1 

- 798 

X 




X 



petroleum 









X 

petty monarch 

1 

- 472 

X 







petty state 

1 

- 419 

X 







phalanx 

1 

- 697 

X 

X 






pharaoh 

1 

- 512 

X 

X 






phenomenon 

1 

- 049 








philanthropy 

1 

+ 122 



X 

X 




phratry 

1 

-1 618 


♦ 



X 



physical proximity 







X 



physician 

1 

+ 616 








physics 

1 

+ 327 


. 






physiocratism 

1 

-1 355 






X 


picket 

2e 




X 

X 



X 

picketing 

1 

- 286 

. 


. 





picture writing 

1 

- 364 

. 


• 





piece rate 

1 

— 755 



X 

X 




piety 

1 

+ 074 






. jr.h.t. 

pilgrim 


.... 

. 






X 

pilgrimage 

1 

+ 327 







• . 

pillage 

1 

+ 067 

X 





X 

X 

pillory 

1 

- 323 

X 






X 

pilot 

1 

+ 495 







a 

pioneer 

1 

-f 442 

, 






5 z 

piracy 

1, 2g 

■f 385 

X 






XX 

plagiarism 


. < . • 






X 

- 

pitched battle 

i 

+ .205 

X 







plains 


. 







X 

plaintiff 

2g, $ 

.... 

X 

X 



X 



plan 

S 

. 







X 

plank 

2g 



• » • 





5 z 

plantation 

1 

+ 522 

. 

. • • 





X 

platform 

1, 2g, S 

+ 558 

X 

X 

X 

X 



• 

plaudit 

1 

- 191 




— 



• 
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Word 

I 

1 

C/9 

M 

12 

Morton Weights 

Wesley List 


Wesley Tests, 
Editions No. 

Wesley 

College 

*2 X ^ 

3 . ^ CO 

s- + S 

c 3 u E 

p 0 . ^ a. 


X 

«-*l ^ 

play 

2i 







pUyground 

2g. S 






X 

playground director 

S 






. r • • 

plea 

1.2g 

+ .316 

X 




Z X 

plebeian 

1 

-.408 

X 




» • » . 

plcbiicite 

1. 2g. S 

-.866 

X 




X X 

pledge 

1 

+ .379 

. 




5 X 

plenipotentiary 

1 

-.887 

X 




X X 

plot 

1 

+ .S00 





X 

plunder 







X 

pluralistic behavior 

E 

»...*• 






plurality 

1.2g 

-.493 

X 


X 

X 

X 

plural voting 

1 

-1.009 

X 





plutocracy 

1, 2g. S 

-.803 

X 

X 

X 

X 

X X 

pocket veto 

1 

-.819 

X 




• m t m 

point of order 

1.2g 

-.603 

X 





point of view 

1 

+ .448 





. . jr.h.s. . 

police 

1. 2g. S 

+ .616 

X 




■ . . y 

police captain 

2g 








police court 

1.2g 

+ .437 

X 





police power 

1. 2g, S 

-.423 

X 





policy 

1. 2g, S 

+ .437 

X 




z 

political 


...... 





X 

political activity 

« . « . 


• 



. • 


political appointees 






» ( 


political institution 

i 

-.324 

X 



• • 


political aspirations 

• < « • 






X 

political district 







X 

political genius 



, 




X 

political liberty 

1 

-.287 

X 




» * • . 

political machine 

1 

-.095 

X 




• • ■ ■ 

political parties 

l,2g 

+ .178 

X 




X z 

political reform 







X 

political rights 

2g 






♦ X , . 

political science 

1 

-.455 

X 




X 

political situation 







X 

politicians 

i,s 

+ .447 

X 




. X 

politics 

If 2g 

+ .315 

X 




z 

poll tax 

1. 2g 

-.128 

X 





polls 

1. 2g, S 

+ .206 

X 




. X z 

polyandry v 

1 

-1.066 





X . . . 

pdygamy 

1 

-.175 




X 

X . . . 

pidyglot 

1 

-1.025 





. • . « 

polytheism 

1 

-.898 

• 

• 
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Word 


I 

g 

'C 

M 

Wciley Testt, 
Editions No. 

Colleitt 

c/} 

M 

c 

o 

u. 

o 

:s 

tl 

s 

- s s a c 

^ ^ 
S £ S 


pomp 

1 

+ .137 

pontificate 

1 

- ,<346 

pooling 

1, 2c. S 

- . .SO? 

pool room 

1 

+ .SOO 

poor farm 

1 

+ .326 

poor house 

1 

+ .326 

“poor white” 



pope 

1 

+ .25R 

populace 

1 

+ .1H 

popular 



popularity 

i 

+ .563 

popular sovereignty 

i 

-.829 

popular suffrage 



popular vote 

1 

+ .189 

Mpulation 

1, 2s. E, S+.326 

Populism 

2g 


Populist Party 

2g 


port 

1 

+ .390 

portage 



port dues 

i 

-.228 

portfolio 

.... 



pose 

1 

+ .016 

posse 

s 


posse comitatus 

2g 


possessions 

1 

+ .558 

post 



postage 

i 

+ .747 

postal air mail 

2g 


postal money order 

2g 


postal savings 

1 

-.049 

postal savings bank 

2g 


postal service 

S 


postal system 

l.s 

+ .385 

poster 

1 

+ .453 

posterity 

1 

-.059 

postmaster 

1, 2g, S 

+ .695 

postmaster general 

S 


post office 

S 


post office (rural free 



deliver) 

2g 


post roads 

1, 2g 

-.117 

postpone action 

.... 


potentate 

1 

— .459 

potential customers 

.... 



s 

X 

X 


Pnessey 
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Word 


potential economic 
demand 
poundkeepers 
poverty 
power 
powerful 
powers 

powers of Congress 

powers of president 

powers of states 

practices 

pragmatic 

prairies 

prayer 

preacher 

preamble 

precede 

precedent 

precept 

precinct 

precipitates 

precision 

predecessor 

predestination 

predict 

predisposition 

predominate 

pre-eminence 

prefect 

prefecture 

preference, selective 

preferential tariff 

preferential voting 

preferred stock 

prehistoric 

prejudice 

prelatt 

premier 

premises 

premium 

preparedness ^ 

pre-primary 

prerequisites 

prerogative 


I 


a 



V 


Wesley 

Wesley Tesu, College 
Editions No. . — — . 


zi i = > a u 


26 


• 

1. 2e, S 

+ .506 


i,s 

+ .448 

X 

1 

+ .448 

X 

ig" 


• 

2g 



26 



1 

+ .385 

X 

1 

-1.235 

X 

r ■ 

+ !448 

• 

1 

+ .616 

. 

1 

-.206 

X 

1 

+ .258 


1. 26, 2s 

-.122 

X 

1 

-.254 


1. 2g, S 

+ .032 

X 

1 

-.006 

X 

1 

+ .016 


1 

+ .274 

X 

1 

-.782 


1 

+ .385 


1, 2s 

-.423 


1 

-.011 


1 

-.074 

X 

i”' 

-1.061 

X 

2s 



1 

-1.035 


1 

-1.072 

X 

• * ■ • 

1 

-!653 

X 

1. 2s 

+ .374 

• 

'l.2g 

-.326 

X 

i',2e 

+ .327 


1. 2g 

+ .327 

X 

i. 2g 

-.692 

X 


X X 


\ 


X 

X 

! 4 


X 

X 

X 


X 

X 

X 



X 


X 


z 

X 

2 

X 


X 

z 

X 

X 


X 
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Word 


presbyters 

pre-school 

prescribe 

presentment 

preserve 

president 

press 

pressure, social 

pretence 

pretension 

prestige 

pretext 

prevention 

preventive medicine 

previous question 

pre-war 

price 

price fixing 
price fluctuations 
price of exchange 
price, wholesale 
prices 
priesthood 
priest-king 
primary 

primary education 

primary election 

primary, non partisan 

primary, open 

primary, preferential 

primary schools 

primate 

prime minister 

primitive 

primitive peoples 

primogeniture 

prince 

princess 

principal 

principality 

principletl 

printing 

printing-press 

prior 


2 

m 

«« 

M 

• 

Wesley Tests, 
Editions No. 

Wesley 

College 


c 




c/a 

t 

o 

1 

^ - > 

a 



^ 1 


S a. 


1 

- 987 





1 

- 248 


\ x 



1 

-(- 201 





2g 






1, 2s 

-h 469 





1, 2g, S 

-1- 617 

X X 


X 


1 

+ 469 




X 

2s, E 






1 

-f- 279 


. • » 



1 

-I- 100 


, 



1, 2s, E 

+ 004 




X 

1 

+ 118 

X 




1,S 

+ 406 

X 




1 

- 381 


, 



1 

- 803 

X 



X 






4 


2e 



, , 



1 

- 703 





1 

- 697 





1 

-1 035 





It 



X X 



1 

+ 305 


. 



1 

+ 1.37 


, 

6 

X 

1 

- 686 

X 

, 

6 


1, 2g, S 

- 117 




a 

1 

-f 095 



. 


S 


X 

X 

. 





X 






X 






K 



1 

+ .148 





1 

- 951 





1, 2g, S 

- 195 

X 



y 

1 

+ 015 




X 




X 


, 

1 

-1 035 


X 


. 

1 

+ 448 

X 

♦ . . • • 


X 

1 

+ 448 

X 


z 

zz* 

i.s 

+ .501 

X 


z 


1 

- 318 

X 



z 

1 

+ .385 

X 

1 » • • • 



1 

+ 690 

. 



• 

1 

+ .690 

. . 



- 

1 

+ .079 

■ ' 




• 
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We«ley Tern, 
Editioni No 


Wedey 

College 


- ^ > 


D PU tid b. 


prion ry 

prison 1, 

prisoner 

privacy 1 

private 1, 

pnvateer 1 

pnvate expenditures 
private ownership 1 

private property 1 

pnvate nghts 1 

pnvate war 1 

pnvilege 1, 

pnvileged classes 1 

probate S 

probate courts 
probation 1 

probanon officer 1 

problem I, 

procedure 1, 

proceedings 

proceeds 1 

process, cultural 2s 

processes of production 1 


1. 2g. S +706 
1 + 443 


\ + 146 

1 + 277 

1 - 292 

1 - 428 

1, 2g, 2s, S+ 316 


procession 
proclamation 
proctor 
procurator 
prodigy 
product, social 
produce exchange 
produce 
production 
productivity of labor 
profession 
professional class 
professionalization 
professionalization of 
industry 
professor 
proficiency 
profit 

profit-sharing ' 

profiteer 

program 


1. 2g. S 
1 
1 
1 

2s, E 
2e 


+ 258 

+ 122 

- 512 
+ 500 
+ 447 

- 259 
-1 250 


+ 506 
+ 506 

- 841 
+ 316 
+ 182 


-1 113 
+ 501 
+ .188 
+ .553 
- 361 
+ 126 
+ .616 
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Word 
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'v 

c 

o 

o 

s 

V 


iVesley Tests, 
Editiotu No. 

M M ^ 

Wesley 

College 

ji 

1 - 
D £ 

b 

S 

(1 

progress 

1.2s, E.S 

+ 448 

X 






X 

progressive 

progressivism 

1 

-.887 

X 




X 


a 

prohibit 

I 

4-.574 

X 





1 


prohibition 

1, 2g. S 

+ fiS8 

X 





s 

z 

project 

1 

■f 2S8 








proletariat 

l.2g 

- 867 







z 

promenade 

1 

+ .085 








prominence 

1 

+ 316 








promiscuity 

1 

- 749 








promise 

promissory note 

1 

- 074 





X 


• 

promote 

i,s 

+ 632 








promoter 

1, 2e 

+ .148 




X X 



X 

promulgate 

1 

- 163 

X 

X 

X 




X 

pronounce 

1 

+ .SS8 

X 

X 






proof 

1 

+ .558 

X 







propaganda 

1. 2g, S 

+ 206 

X 

X 

X X 

X X 

x 


a 

propagate 

1 

+ 064 








propertied class 

1 

-.059 








property 

1, 2s, S 

+ 501 





X 


y 

property tax 

1 

-.1% 

z 







prophet 

1 

+ .332 








proportionate 

1 

+ .127 

« 






k 

proportional repre- 
sentation 

1 

-.928 

X 






* 

proposal 

1 

+ .368 

X 






a 

proposition 

1 

-f" 368 

X 







proprietary article 

1 

-.934 





X 



proprietary company 








proprietary province 

1 

-.987 

z 







proprietor 

1 

+ .321 

• 





t 

z 

proscription 

1. 2g, S 

+ .322 







X 

prosecute 

X 

X 





X 

prosecuting attorney 

1 , s 

-.059 

X 

X 




» 

a 

proselyting spirit 

2s 







r 

« 

pro-slavery 


• 






1 

X 

prosperity 

l.s 

+ .506 






t 

a 

prostitute 

1 

.090 







- 

protagonist 

1 

-.928 







• 

protection 

1, 2g, S 

. SOb 

X 






X 

protection of citizens 
abroad 

2g 

-.824 




, , 




protectionism 

1 

• 



• 


• 

* 
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a 

■a 


Word 


1 

e 

■a 

•t 

Q 

c/) 

w 

0 

m 

.2 

0 

z 


Weiley Testi, 
Edition! No. 


Wesley 

CoIlcKe 


M 


^ > 
M M l-N ^ 


. V s 

-§35 

a, a. 


protective 

. . . 

, , 

. . . . 


protective tariff 

I 

— 

250 

, 

protectorate 

1> 2g 

— 

.771 

X 

protest 

1 

+ 

.500 

X 

Protestantiim 

1 

+ 

.052 


protocol 

1, 2g 


.861 

X 

province 

1 

+ 

.148 

X 

provincials 

1 


.370 

X 

provincial government 1 

— 

.713 


provision! 

1, s 

+ 

.495 

X 

provisional government .... 



X 

proviso 




. 

provocation 

1 

+ 

.063 

X 

prowess 

1 


.053 


proxy 

psycWanalysis 

1 


.391 

X 

psychology 

1 

— 

.005 


psychopath 

1 

-1 

.065 


public 

l.s 

+ 

.389 


public affairs 

1 

+ 

.327 

X 


public building! 2e 

public control of cable 2e 
public control of power 2e 


public control of radio 

2e 

public control of 
telegraph 

2e 

public control of 
telephone 

2e 

public defender 


public domain 

ie, 2g 

public education 

2g 

public health 

1. 2g, 2e 

public health, 
quarantine 

E 

public industry 

2e 

public land 

2g 

public law 

2g 

public library 

1 

public management 

2e 

public managcincnt and 
DiiratS Pwrtership 
pubKc management and 
pslfie ownership 

2e 

2e 


+ 


253 


558 


X 

X 


X 

z 


X 

X 

z 

X 

X 
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Word 


public opinion 
public ownerthip 
public property 
public regulation of 
private enterprise 
public school 
public speaking 
public spirit 
public, the 
public utility 
public welfare 
public works 
publicity 
punish 
punishable 
punishment 
punitive expedition 
purchases 
purchasing power 
pure democracy 
pure food law 
puritan 
puritanism 
quaestor 
quaker 
qualifications 
qualified veto 
quantity theory 
quantity theory of 
money 
quarantine 
quarters {v ) 
quasi'public 
queen 
queer 
quell 

question (public) 

quibble 

quit rent 

quorum 

quota 

rabbi 

rabble 

race 


List Symbol 

Morton Weights 


Wesley 

Wesley Tests, College 

Editiont No. ' — 

Ck. 

1 ^ ^ i ^ 1 

l.2g.2s, 

S+ 252 

X 

. • • • 

X 

I. 2g. 2e 

+ 132 

X 


. 

1 

+ 390 

X 

. 

• 

2e 



• • 

, 

1 

+ 574 


. * > • 

» 

1 

+ 453 


. * • • • 


1 

+ 269 

X 



• 

2t. E 



. 


L 2g, 2e 

- 233 


X X . X . 

X 

2g. 2e 



X 

. 

2g. 2e 



X 

• 

1.2g 

+ 264 


, . . . « 

y 

S 


X 

. . . s • 

s 

S 


X 

» t » 

. 

1.2g 

+ 616 

X 

X . . X . 

X 

1 

- 644 

X 

X 





t » • • 

X 

1 

- 380 


1 . . • s 





X 


1 

+ 057 

X 

. XX... 






X 

1 

- 556 


, p • * • • 

. 

1 

-1 566 

X 


. 





X 

i.s 

+ 385 

X 

X X 

• 

1 

-1 019 

X 


> 

1 

-I 082 




• 

2e 




• 

2g S 


* 


X 

1 

- 392 

X 


X 

1 

- 772 

X 


s 

1 

- 632 


. • • . . • * 

X 



X 

• »••••** 

• 

i 

+ 148 

X 


- 

s 




• 

1 

■4* 0Z6 



• 

1 

-1 061 



• 

2g,S 


X 


X 

1 2g 

- 121 

X 

zx»x> 

X 





z 

l' 

- 027 



X 

l,2s,S,E + 501 

X 


X 
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Word 

J 

s 

JC 

■t 

■J 

Wesley Tests, 
Editions No. 

Wesley 

College 


t 

M 

e 

g 

o 

S 

& 



III 

IV 

Unpub. 

Pub. 

Keity 

Pressey 

race balance 

It 








race conflict 

\ 

-.054 







race inequality 

2i 








race pi^che 

2i 

-.387 







race suicide 

1 




X X 

X 


racial 

1 

+ .099 







racial diflFerencei 

2s 








racial minority 

1 

-.714 

X 






racial prejudice 







X 


radiation 

2i 








radicalism 

1 

-.527 

X 




X 

z 

radio 








X 

raid 

i 

+ .379 

X 





X 

railroad combination 

2e 





X 



railroaded 







X 


railroad franchise 

It 








railroad rates, com- 
petitive 

2e 






• » e 


railroad rates, export 

2e 








railways 

S 

-.771 





X 


rajah 

1 

X 




X 


rallv 

l>2g 

+ .495 

X 






rank 

1 

+ .501 

X 




z 


ranks 








x 

ranking member 

2g 

+ !442 







ransom 

1 







rapacity 

1 

-.608 







fppine 

I 

— .766 







rapport 

E 

+ !i32 







rate (interest) 

1. 2g,S 





X 

X 

ratification 

1. 2g, S 

+ .190 

x 




X 


ratify 

S 

+ .195 




X X 


z 

ratio 

l.S 

X 




X 


ration 

1 

+ .258 







rationalism 

1 

-.935 







rationalization 

2s, E 






X 


ravage 


+ .327 






X 

raw materials 

i 





X 

z 

reaction 

1, 2s, E 

+ .131 

X 




X 

* 

reactkmt drc^r 

2s, E 







- 

leactioaanes 

1 

— .333 

X 

X 



X 

X 

reacdaiism 







X 


readings of a bill 

2g 

+ .079 





X 


readjustment 

1, 2s 





X 

• 
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I 

E 

m 

£ 

c 

if 

w 

12 

Wetley Triti, 
Editiotii No. 

Wetlfy 

CotlcKe 

t/i 

w 

m 

c 

0 

1 
z 

t 

■a 

AI 

HI 

KII 

II 

1 

-d ^ v 

1 i 

&2 a. 


rca^-madt 

1 

+ 49S 





reaffirmation 

1 

- 201 


. 



real estate 

1 . s 

+ 4'>'; 



. jr.h.s 


realm 

1 

■f 

X 



X 

real property 

1 

- ^8) 


■ • • 



rcapportionment 

1. 2g 

- 37*; 

X X 




rebates 

1 

- 




X 

rebel 

1,2k 

+ IbS 

X 


t 


rebellion 

1.2k 

+ ^06 

X X 



X 

rebuff 

1 

4- 085 





recall 

1, 2r, S 

+ 085 

X 



X 

recalcitrant 





X 


recant 

1 

- 433 


, , 



recapture 




« » > • 


X 

receipt 

1 

+ 432 

. 

X 

X 


receiver 




X X 

X 


receivership 

2e 


, 

. . 

» • 


reception 

1 

+ 568 

X 


X 


recess 

1 

+ 553 

X 


• » 


recession 




, , 

X 


reciprocal 

i 

- 449 



. 

X 

reciprocity 





X 


reciprocity treaty 

1. 2e, 2g, 2 

s— /77 

X 


X 


recite 

1 

+ 553 



, , 


reclamation 

1. s 

- 228 

, 

. . • 

X 

z 

recognition 

1, 2g, 2s 

+ 368 

X 

X 

. 5 


recommendation 

1 

+ 626 

X 


X 

X 

recompense 

1 

+ in 


. 



reconciliation 

1 

+ 200 

X 


X 

X 

reconnaisance 

I 

— 166 



• 


reconnoitre 





s 

s 

re conquest 

1 

+ 073 



» • 


reconstruction 

1, 2g, 2s 

+ 306 

X 



1 

records 

1, 2g. S 

+ 558 

X 


X 

y 

recreation 

1, 2s, S 

+ 568 




X 

recreational 





X 

• 

recreational facilities 

, 




X 

. 

recruit 

1 

+ 441 

X 


. 

X 

recuperation 

1 

+ .079 

• 


• 


recurrence 



. 


x 

S 

redeem 

1 

+ 269 

X 


. s 

t 

redemption 

1.2g 

+ 142 

. 


• 

. 

redint^ration 

E 


• 


X 

• 

rediscount 

1. 2e 

- 739 



• 
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TESTS AND MEASUREMENTS 


Word 

List Symbol 

Morton Weights 

Wesley List 


Wesley Tests, 
Editions No. 


Wesley 

College 

Kelty 

Pressey II 

Unpub. 

Pub. 




> 

redistribution 

1 

-.248 









redoubt 



, 







X 

redress 

i 

-.089 

X 








redress of grievances 

1. 2g 

-.507 

X 








"Reds" 

1 

-.523 

X 







X 

red-tape 

1 


X 



X 

X 




reduce 

1 

-1-.506 

X 






X 

X 

re-education 

1 

-.575 









re-election 

1. 2g. S 

-f-.369 

X 






z 

X 

re-enact 

1 

-1-.206 

X 








reenforcements 

1 

-t-.311 








z 

referee 








X 



referendum 

i. 2 g. s 

-.522 

X 

X 




X 

X 

z 

reforestation 

1> 2g, 2e 

-.052 









reform 










2 

reformation 

I, 2g. 2s, S 

-f-.315 








X 

reformatory 

1 

-1-.385 









reformer 

1 

-1-.326 

X 



X 

X 


z 


refugee 

1 

-l-.Oll 

X 







X 

refusal 

1 

-I-.632 









regency 

1 

-.514 

X 

X 






X 

regicide 

1 

-.730 

X 

X 

X X 

X 

X 

X 



regimented 

1 

-.707 

X 








regime 

1 

-.566 

X 







z 

regimen 

1 

-1.045 









regiment 



. 







X 

region 

i 

-f* ,632 

. 







y 

register 

2g^ 


. 







X 

hgistration 

i.s 

-I-.316 

X 

z 





X 


regress 

2s, E 


• 







. 

regressive 



. 





X 



regulan (army) 

i 

+ .322 

X 







X 

regularity 

1 

+ .458 

X 








regulation 

li 2g, S 

+ .458 

X 






X 

X 

rehabilitate 

1 

-.259 








X 

reichstag 

.... 









X 

reign 

1 

+ .385 

X 







X 

reinstate 

1 

+ .148 

X 







X 

reinsurance 

1 

-.349 









reiecdon 
rdapse ' 

1 

+ .574 

• 





X 

X 

X 

relatiottt 

i 

+ .637 

X 







X 

relationships 

2s. E, S 


X 








relatives 

— 


• 





X 


• 
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lelax 1 

relict 1 

relief 1 

relief expedition 1 

religion 1. 2t 

religioui diisemination I 
religious freedom 1 

religious lawt 1 

religious toleration 1 

relincjuith 1 

remains 1 

remand 1 

remnant 1 

remodel 1 

remonstrance 1 

removal 2g 

remunerative 
renaissance I 

rendition 2g 

renewal I 

renomination 2g 

renounce 1 

renovate 1 

renown 1 

rent U S 

renunciation 1 

reorganization £> 

reparations li 2| 

repatriate I 

repeal 1> 2j 

repel I 

repentance I 

replacement 1 

replevin 2g 

report It S 

report a bill 2g 

representation li 2( 

repietentation, direct 
representation, factional . . . 
representation, minority 
representation, poliacal . . 
representation, popular . 
representation, propor- 
tionate 


+ 127 
+ 

+ 127 
-H 417 

- 910 
+ 073 

+ Oil 
+ 127 
+ 685 

- 798 
-1- 458 
+ 579 
+ 327 


- 593 


458 X 

+ 226 X 

-I- 269 
+ 342 
-I- 379 

- 015 

- 300 X 

- 797 X 
, S +516 X 

+ 453 X 
+ 326 
+ 232 X 

+ 569 X 

.2i,S+ 390 X 
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Word 

3 

e 

8 

■s 



Wesley Tests, 
Editions No. 

Wesley 

College 


B 

w 

.n 

1 

s 




III 

IV , 

Unpub. 

Pub. 

Kelty 

Presscy 

rcprcKntative 

2g, S 

+ .074 

X 





X 

repretentative aiaembly 1 

X 





• 

repretentative-at-large 

repretentative 

1 

+ .153 

• 




X 


government 

X 



X 



repreition 

1. 2s, E 

+ .067 

X 






reprieve 

2b 


X 




X 

. 

repriiali 

1 

-.397 

X 





X 

reproduction 

1 

+ .322 

, 





, 

republic 

1, 2g, S 

+ .310 

X 

X 




z 

Republican Party 

2g 

+ .127 






z 

repudiate 

1 

X 



■ , 


ZX 

requeit 

1 

+ .626 




• » 


, 

requiiition 

!• 2g 

+ .016 

X 



X 


X 

rescind 

.... 


, 





X 

research 

1 

-.116 




• • 

X 


resentment 

1 

+ .442 




» * 



reservations 

1.2g 

+ .057 

X 



• • 

X 

X 

reserve 

1 

+ .322 

X 




X 


reserved powers 

i 

- J79 

X 



• 



reserve corps 

X 



• • 



reserve fund 

1 

-.323 




• * 



reserves 

2g, 2e, S 


X 



» * 

X 

X 

reserves, accumulated 

• . » . 






• 

X 


re-shaping 

2s 








resident 

1. 2g, S 

+ .SS8 

X 

X 

X 



X 

residual 





• • 

X 


resignation 

1. 2g 

+ .632 

X 



• » 



rosistance 

1.2g 

+ .516 

X 




. jr.h. 

S. X 

resolutions 

1. 2g 

+ .553 

X 

X 

X X 

X 

X 

X 

resources 

i.s 

+ .443 

X 



■ • 

X 

x 

jespect 

1 

+ .569 

X 



• « 



response 

2s, E 





« K 



reaponsible government 2g 

-.887 







responsible ministry 

1 

X 




X 


responsibility 

1, 2s, S 

+ .385 

X 

X 

X X 

X X 

X 


resuurant 

1 

+ .747 

, 






restitution 

1 

-.445 

. 






resdeaaness 

2s, E 





• ■ 



restoration 

1 

+ .183 

X 



• * 

X 

X 

restraint c£ tr^de 

1 

-.634 




■ • 


• 

restriegiMt 

1 

+ .385 




■ s 


z 

result 

. • * . 





• % 

. . 4 

* 

resumption 

1 

+ .127 

• 



• • 

. 

X 
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Word 


i 

E 

.a 


m 



Q 

s 


I 


Weiley 

Wctiey Teiti, CoUet* 
Ediaoni No. - — — * 



jO 

I 

e a 
U A. 


fc> 

2 


I 


reciirrcction 

retail 

1 

+ .211 










X 

retail phcca 

i 

+ .258 





X 

X 





retailer 

1 

+ .267 











retainer 

retaliation 

1 

1 

-.191 
+ .072 

X 









X 

retardation 

2i 











retirement 

1 

+ .378 











retreat 

1 

+ .572 

X 









X 

retrenchment 

1 

-.398 

X 

X 






X 



retribution 

l,2i 

-.369 











retrieve 

1 

-.362 











retroactive 

1 

-.791 

X 

X 






X 



retrograde 

1 

-.640 










a 

retrogreti 

1 

00 










• 

retumt 

h 











X 

reunion 

l 

+ .S11 

X 








X 


revaluation 

1 

+ .061 











revelation 

l,2i 

+ .073 











revenge 

l.2i 

+ .578 









5 


revenue 

1. 2g. S 

+ .328 

X 

X 






X 


X 

revenue, anticipatory 
revenue, internal 

2e 






X 

X 





1, 2e, 2g 

-.263 

X 

X 









reverence 

1.2i 

+ .323 








• 



reverKC 

1 

+ .184 

X 







• 


X 

reviiion 

l.S 

+ .184 

X 




X 





X 

revival 

1.2« 

+ .183 











revocation 

1 

-.325 

X 










revolt 












1 

revolution 

l,2g, 2*,S+.395 

X 

X 







4 

1 

revolutionary 










X 


• 

revuUion 










X 


« 

reward 

1 

+ .573 

X 






X 


4 

a 

rider 0**-) 

1 

-.781 

X 

X 






X 



ridicule 

1 

+ .439 











“Right" (n. pol.) 

l.S 

-.898 

X 









• 

right of bdligcrentt 

1 

-.981 

X 










right of depoeit 

1 

-1.003 











right of neutralt 

1 

-.897 

X 









a 

right of recall 


-.742 








X 


a 

ript of aearch 

i 

X 







. 


• 

right of way 

2e 






X 

X 


• 

» 

a 

rigid ctasaea 

.... 







• 


X 

■ 

f 

ring (pol.) 

S 

...... 

• 





• 


X 

• 

X 


TESTS AND MEASUHEMBNTS 


Word 

J 

1 

m 

Morton Weights 

• B 

1 
» . 

Wesley Tests, 
Editiont No. 

Wesley 

College 

JO ^ 

3 . a 

5- -r S 

::j n. uS 0. 

AI 

III 

Mil 

II 

I 

rinKleader 

1 

+ .389 

X 



not 

1 

+ 517 

X 


z 

riparian right! 





X 

risk 

1. It 

+ .573 


X X 

X 

rite 

• » . < 




X 

ritual 

1 2t 

-.078 




rivalry 

l,2g, 2s, E-I-.4S1 

X 


X 

road surveyor 

2g 





road tax 

1 

-1.003 

X 



robbery 

1 

+ .573 



. 

roll-call 

1.2g 

+ .5556 

X 

X . . . , 


rolling stock 





X 

romance 

.... 




X 

romantic 

.... 


. 


X 

rostrum 

1 

-.2^ 

X 



rotation 

I 

+ .123 

X 


X 

rout 

1 

+ .139 

X 


■ * i • 

routes 

1 

+ .506 

X 


X 

rowdies 

1 

+ .389 



. jr.h.s. . 

royal 

1 

+ .456 

X 


X 

royal ceremonies 

• » . . 




X 

royalists 

1 

+ .043 

X 



royalty 

.... 




X X 

rudimentt 

1 

-.096 



• • • • 

ruins 

1 

+ .573 




rule of merit 

. . . • 




X 

"rule of reason” 

2g 




X 

ruler 

1 

+ .573 

X 


X X 

sules 

S 





rum running 

1 

+ .456 




rural 

1 , s 

+ .389 




rural free ddiveiy 

2g 




X 

ruthless 

1 

+ .206 

X 



sabotage 

1 2e 

-.815 



X X 

tack 

1 

+ .328 

X 



sacramentt 

1 

-.637 



• • • • 

sacred 

1 

+ .328 




sacrifice 

1 

+ .389 



■ « • • 

sacrilege 

1 

-.341 




safe deposit box 

1 

+ .511 




safeguard « 
sahiD 

1 

+ .328 

X 


X 

X X 

safety legislation 

U 





sailor 

— 




X 
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Word 

List Symbol 

3 

C 

e 

§ 

0 

5 

m 

K 

■2 

Wesley Tests, 
Kditions No. 

- ^ > 

Unpub 

Pub 

Keity 


saint 

1 

+ ns 






salaries 

1 . s 


X 




z 

sale 

1 




X 


X 

salesman 

1. 2e 

+ ^73 


X 




salestax 

1 

- 478 

X 





salient 

1 

- 663 

X 

X 




saloon 

1 

+ 328 




. 

X 

salute 






. • 

X 

salvation 

1 

+ 328 






salvo 







X 

Samurai 

1 

-1 314 






sanction 

1.2s 

- 169 





ZX 

sanctity 

28 







sanctuary 

1 

— S63 






sane 

1 

+ 517 






sanitation 

l,S 

+ 317 




s 


satrap 

1 

-1 071 

X 

X 




satellite 

1 

— 753 

X 

X 


. 


saturated 






X 


savages 

1 

+ 578 





X 

savant 

l 

- 920 






savings 

l.s 

+ 573 




X z 

s 

scabbard 







X 

scab 

1. 2e 

- 157 


X X 


X 

y 

scalawag 







X 

scandal 

1 

+ 511 






scarcity 

I 

+ 451 





» 

scepter 

1 

— 133 

X 



• 

■ 

schedule 

1 

+ 450 

X 



X 

* 

scheme 

1 

+ 511 





• 

schism 

1 

- 793 





• 

scholar 

1 

+ 311 



. 



scholarship 

1 

3{>8 




• 


scholasticism 

1 

-1 108 






schools 

i.s 

+ 672 




• 

X 

school board 

1 

+ 558 




X 


science 

1 

+ 253 




X 


scientific management 

1, 2e 

- 597 




• 


scientific research 

2s 





• • 


scolding 

1 

+ 689 




• • 


scout 








scutage 

1 

-1 429 

X 

X 

• 

• 


sea-level 






• • 

X 

sealed legal 




• 

• 

X 
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Word 

3 

c 

5 

■6 

s 

*0 

Wesley Tests, 
Editions No, 

Weslev 

College 



X 

ly) 

40 

G 

S 

0 

S 

•a 

4^ 


A1 

III 

Kll 

II 

Unpub. 

Pub. 

Kelty 

w 

1* 

!■ 

Ow 

sealer of weights and 
measures 

2g 








seaport 

1 

+ .384 






X 

sea-power 

1 

+ .09S 

X 






seasonal 

1 

+ .043 







search-warrant 

1 

-.083 

X 




X 


seat 








X 

secede 

1.2g 

+ .264 

X 




y 

z 

seclusion 

1 

+ .326 







secondary schools 

1 

+ .379 







second-rate power 

1 

-.287 

x 






secrecy 

2s 







, 

secret ballot 

1 

-.214 

X 






secret diplomacy 

1 

-.529 

X 






Mcret service 

1. 2g 

-.052 

X 





X 

secret society 

1 

+ .379 




X 


, 

secretariat 

1 

-.718 

X 





, 

secretary 

2g. S 



X 





z 

sect 

1 

- 143 





6 

z 

sectarianism 

1 2s 

-.627 

. 





X 

section 

S 


X 





y 

sectionalism 

1. 2s, S 

-.297 

X 

X 





sector 

1 

-.289 

X 






secular 

1 

-.582 






X 

securities 

1 

-.058 







security 

1, 2s 

+ .390 

X 





X 

sedition 

l,2g,2s,S 

-.333 

X 

X 




z 

see (papal) 







X 

segregation 

i, 2s, E 

-.432 




X 



seignorage 

2e 

+ .689 




X 



seize 

1 






X 

selection 

2s, £, S 








selective draft 

1 

-.381 

X 






selective exclusion 

• • • 





X 



selectman 

1.2g 

-.676 

X 

X 

X . X . 



X 

self 

£ 








self^ntrol 

1 

+ .390 





5 


self<onsciou8 behavior 

2s 








selMefeose 

1 

+ .448 







sdMenial « 

1 

+ .385 





X 


selMetennination 

1.2g 

-.559 

X 

X 





selfoducation 

I 

-.017 







sdl^vemment 

1. 2s, 2g 

+ .006 

X 




X 

z 

sdf, expanded 

2s 


- 





• 
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Word 

1 

g 

-w 

*6 

'C 


Wesley Tests, 
Edtdons No. 

"iTIl 

Wesley 

Gdlege 

C/3 

w 

« 

U 

1 

1 

I 

II 

1134 

rii 

IV 

14 1 1 

self, mirrored 
leir preservation 

2i 

1 

+ .19S 




. 

self reliance 

1 

+ .201 



. . 6 . 

self sacrifice 

2f 




self tuftcient 

1 

-.226 




self Buftcient nation 

It 





self support 

1 

+ .327 



» • s 

tell 

1 

+ .747 




fenate 

1. 2«. S 

+ .322 

a 


. . 5 > 

senator 

2g,S 




a 

senatorial courtesy 




a 

leniorage 

.... 




X 

seniority rule 

2g 




X 

sensation 

1 

+ 437 



. . jr.hj, , 

sensuality 

1. 2s 

-.122 



• . s * 

sentence 

1 

+ .558 

X 


X X 

sentiment 

1 

+ .253 

X 


BBSS 

sentinel 

1 

+ .443 

X 

X X 

X 

separation 

1 

+ 558 

. 


x 

separatists 

i 

-1 114 



X 

sepoys 

X 

X X X X X 

X 

sequester 


-1.476 



x 

seraglio 

1 

, 



serf 

1 

-.528 

. 


X 

serfdom 

1 

-.528 




sergeant 

1 

+ .237 

X 


X 

tergeant'at'armt 

2g, S 

+ .627 

z 


X 

sermon 

1 

. 


t 

servitude 





X 

servant 

i 

+ .616 

. 


X 

service (army) 


+ .458 

. 


s 

services 

l.s 

z 


X 

servile 



. 


X 

sesame 

1 

-l.OSO 

. 


. 

session 


-^269 



X X 

sessions 

i, 2 g, s 

X 


* s * • 

session, long 

If 2g 


x 

« • s * • 

• • • s 

session, short 

If 2g 

+ .342 

X 

• • . • • 

• • s s 

setback 

1 

X 


4 « * • 

settlement 

.... 


e 

. . . • . ■ 

X 

settlement house 

1 

-.312 

. 


• s • « 

settlementt 

l.s 

+ .427 

X 


t * >» « 

. jr.b.s. . 

sever 

1 

+ .274 



sex 

1 

+ .574 

• 



. 
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TESTS AND MEASUREMENTS 




«• 



Wesley 



5 

c 


Wesley Teif*# 

College 


Word 

3 

Editions No. 

. — ■ — . 



e 

§ 

o 



3 

S 


nv 

M 

4» 

^ > 

& -i 

TJ t 


Zj 




:d a. 

U1 a. 

fhacklM 

1 

+ .211 





•hame 

1 

+ .616 





shareholder 

shares 

l,2e 

-.047 

• 


X 

X 

sheriir 

1, 2g. S 

+ .358 

X 

X X 


X X 

shield 

1 

+ .458 

X 




shilling 

ship-building 

l' 

+ ;S69 

• 



X 

shipment 

1 

+ .532 





ship-money 

1 

-1.065 

X 




ship owner 

1 

+ .385 





ship-subsidy 

1 

-.893 

. 




shipping 

shippins company 

1 

+ .269 

• 


X 

y 

shire 

1 

-.625 

X 




^ogun 

1 

-1.382 

X 

X . . , . 



shop 

2e 

. ..... 





shop dosed 

2e 






shop, committee 

2e 






shopkeepers 

1 

+ .579 





shopping goods 
shop, open 

2e ' 




X 


short ballot 

1 

-.798 

X 


X 


show 

1 

+ .600 





shrine 

1 

+ .137 





siege 

1 

+ .458 

X 



E Z 

signal 

1 

+ .495 

X 




Stature 

1 

+ .632 

X 



X X 

significant 

E 






signor 

silver, bullion 

ie* 





X 

silver certificate 

2e 






silver dotlars 

2e 






simian 

.... 




X 


simony 

1 




X 


simulation 

2s, E 






sinecures 
sinde tax 
sinking fund 

1 

1. 2e. S 
1.2e 


X 

X . . . . 

X 


sin ^ 

1 . 






site 

1 





X 

situation 

E 






skepndsm 

skin 

1 

1 





e • 
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Word 


J 

E 

u 

t/2 



§ 

C 


•I 

V 


Weiley 

Wnley Tettt, Coltcfe 
Editioni No. • — • — ■ 




a 

U> 0 . 


I ^ 


skipper 

1 








skirmish 

1 

% 






X 

slackers 

1 

% 







slander 

1 

X 




X 



slaughter 

1 

X 







slave 








1 

slave trade 

1 

X 






X 

slave holder 








X 

slavery 

2s 

X 




• 


X 

slay 

1 





s 

B 


slogan 

1 

X 


1 


a 


X 

slums 

1 







X 

small farmer 

1 






S 


smunling 

1 

X 

X X 

X X 


K 

• 

a 

snoboery 

1 



, » 





sociability 

2s 




X 




social 

E, S 







X 

social achievement 

2s 



• • t 




• 

social activity 

2s 



» • 





social adaptation 

1 



• « s 




s 

social adjustment 

2s 



s * • 


« 



soaal aggregation 

2s 





« 


s 

soaal ascendency 

2s 



• » • 




B 

social assimilation 

2s 



• • » 




» 

social attitude 

2s 



• » « 



• 

• 

social authonty 

2s 

. 


• ■ • 



» 

• 

social automatism 

2s 



t 

X 



« 

social balance 

2s 






• 

• 

social case work 





X 


B 

• 

social center 

S 






t 

• 

social change 

2s 

• 





B 

# 

social choice 

2s 

• 



• 


B 

• 

social control 

1, 2s 





S 


B 

social discontent 

1 



» ■ 

* 


• 


social disintegration 

2s 

. . * 


* • • 


• 

• 

• 

social distribution 

2e 

• 


» • • 


• 

• 

« 

social equality 

1 

. 


V « • 


• 


0 

social evils 


• 


• • 

X 

s 

• 

0 

social evolution 

1, 2s 



• • • 


• 


• 

social feeling 

2s 



f • • 


♦ 


• 

social fermentation 

2s 

* • • 


s * ■ 




• 

social flexibility 

2s 

• • • • * 



* 



0 

social genesis 

2s 

► • 


> • a 




• 

social group, primary 

2$ 


- 

• * • 

* 
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Word 

1 

1 

W 

M 

□ 

fl 

1 

o 

s 

'•2 _ 

Wesley Tests, 
Edidons No. 

Wesley 

G>llege 

M 

3 

s 

D o. 

^ s 

TB £ 
CL 

4» 

1 

^ - 

AI 

111 

Mil 

II 

Mcial groups, secondary 2s 






social heritage 

2s 

...«•« 



X 


social instability 

2s 






social instinct 

2s 






social institution 

1. 2s 




X 


social insurance 

1 







social integration 

2s 






social interest 

2s 






social justice 

1.2s 






social lags 





X 


social life 

2s 






social manipulation 

2s 






social mechanism 

2s 






social mind 

2i 






social mobility 

1 

• ••••• 





social movement 

2t 



• • • • 


t * 

social nexus 

• » • « 




X 


social order 

2s 



• • • • 


■ » 

social organism 

2t 




z 


social organization 

2t 






social ossification 

2t 






social pathology 

1. 2s 




X 


social pattern 

1. 2s 




X 


social polity 

2t 



» • • » 



social position 

1 






social pressure 

2s 






social problem 

2s 






social processes 

1. 2s 






social product 

2s 






social progress 

2s, 2e 






social psychology 

1 




X 


social purposes 

2s 






social reactions 

2s 






social reform 

1, 2t 






social regeneration 

2s 






social rrauladon 

2s 






social teTadonship 

2s 





■ • 

social ritual 

2t 





• • 

social sanction 

2s 



• • • ■ 


• • 

social science 

1 



* • * • 



social selection 

1. 2t 



• ■ ■ ■ 



social adf<onsciousness 2t 






social service 

... * 





X 

social situadon 

2t 





• 
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Word 

List Symbol 

S 

'S 

S 

V 

to 

0 

s 

** 

1 

Wesley Tests, 
Editions No. 

Wesley 

CoUege 

1 

- 5 = 

t - 1 

6 a o 

£ 0. M 

•ocmI tpint 

2s 






■octal lubility 

1 






■octal aufnation 

2i 






■octal sutui 

2t 






■octal ttratiBcatioa 

2i 






■octal atnicture 

2s 






■octal iuggestioii 

2s 






■octal aurplui 

2e 






■octal aytnpathy 

2s 

» » . 





■octal trend 

2s 

. • . 





■octal type 

2s 






■octal unifotmitiei 

. • < 




X 


•octal unity 

2i 






■octal unrest 

2s 






social valuation 

2s 






social wants 

2s 






social waste 

2t 






social well-being 

2s 





• 

social will 

2s 


, , 




social worker 

1 






socialisni 

li 2g, 2e 





X 

socialist 

S 





a 

socialist party 

2g 





• 

socialtzation 

2i, E 


. 




socieui 

E 






society 

1, 2s, E, S 





■ 

society, civil 

2s 






society column 

1 

. . 





society, feudal 

2s 






society, pastoral 

2, 2s 





■ 

society, political 

2s 

. • 




s 

■oaety, primitive 

2s 





• 

society, static 




X 

X , X . 

• 

society, sympathetic 

2s, E 





• 

■ode^, tribal 

2s 





• 

■ociocracy 

2s, £ 






■odological research 





X 

• 

sociology 

1, 2s. E 

. 


. 


• 

aodo-mcchanisms 




. 

X 

• 

■ocKMphere 

E 





• 

aodus 

2s. E 

. . . 


. 


• 

aiddien 

1 


a 

X . . . 


X 

aolidtation 

1 



. 


* 

aolidtor 

1, 2g, S 

. . . 

X 

* 


X 
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Word 


£ 

e 

a 

•& 

s 


Wetley Test*. 

Wesley 

College 

m 

12 

Editions No. 

E 

X 

M 

1 

0 

S 

& 

AI 

III 

Mil 

II 

I 

Unpub. 

Pub. 


“Solid South” 

2g 

solidarity 

1, 28. E 

solidified 

solicitude 

2s 

solvent 

sophistry 

1 

sorority 

1 

sound money 

I 

source 

1, S 

source documents 
sovereign 

1. s 

sovereign people 

2g 

sovereign power 

2g 

sovereign state 

2g 

sovereignty 

1. 2g, 2s, S 

soviets 

1. s 

spare time 

1 

speaker 

2g. S 

speakeasy 

specialist 

1 

specialization 

1, 2s 

special interests 

1 

special le^alation 

1 

special privilege 

1 

specialty goods 
specialty shop 

1 

specie 

1 

specie payment 

1 

^eciiic 
specific duty 

1 

spectacle 

1 

speculation 

1, 2e, S 

speculator 

2e 

spendthnft 

1 

sphere of influence 

1. 2g 

spies 

1 

spirit 

S 

spiritism 

spiritual 

1 

spiritual qualities 

. . . 

splendor 

1 

split 


spoilsman 

1 

spoils system 

1. 2g, S 


X 


X X 


X 

X 


X 

X 


X 


X 


X 

X 


X 


X 


X 

X 


X 

X 


X 


X 


X 


X 


X 

X 


X 


X 

X 


z 


y 


X 


X 


X 


X 


X 
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Word 


ipolcesman 

iport 

ipum 

(punout value 

•py 

squabble 

squad 

squadron 

squalor 

square deal 

squatter 

squire 

stable 

stable currency 

stability 

stabdization 

Stabilized dollar 

stadt-holder 

staff 

stagecoach 

stage of culture 

stagnation 

stake 

stamp 

stamp tax 

standard 

standard, gold 

standard of conduct 

standard of living 

standardized dollar 

standardized tests 

standing army 

stand-patters 

staple 

starvation 

state 

state aid 

state bank 

statecraft 

state equality 

statehood 

state (of flux) 

state of war 

lUte paper 


5 

E 


1 


2 


Weslev 

M 


Wesley Tests, College 


J; 

Kditions No. - — ’ — 

1 

0 

1 

.A 

^ 1 -i 

5 








X 


X 

1 . . 

X 

lx . . 

1 . . . 

z 


1 

1 

X 

X 

1 

X 

X 

1 

1 


% 

TC . 

1 

X 

X 

1 

2e 

- 386 

a 

1 

- 1 2f^y X 


1 

+ 269 X 

X 

X 

1 

- 


1, E 

+ 269 

. 

1 

-1- 49S 


1 

2g 

5^8 

• 

1 2e, S 
2e 

+ 269 

' 

1 

- 132 


1, 2e, 2s 

- 314 

z 

2e 


X 

1 

-f 322 X 

X X 

I 

- 074 

y X 

1 

+ >69 


1, 2g, S 

-b 448 X 

X t 

1 

- 307 X 

. 

1 

- 281 

t • 

1 

- 385 X 

• • 4 ■ 

2g 


• • • • 

S 



1 

- 548 

. 

2g 

+ 0005 X 

. 

1 

- 375 X 
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Word 


2 

E 

tA 

n 


I 




;2 


Wedey 

Wedey Tests, Coltege 
Edidons No. - — ■ — ■ 



3 

a 

c 

3 




& s 


■3 3 t 
a. :d 0 . 


state’i evidence 

2g 








X 



statesman 

1. 2g. S 

+ .195 

X 








y 

statement (bank) 

1 

-.005 








x 


state religion 

1 

-.466 

X 

X 








state righu 

1 

-.481 

X 








z 

state socialism 

1 

-1.019 

X 









the state 

is 











station 

1 

+ .616 








z 


sutionary 

1 

+ .505 










sutionarv population 

• < * • 








x 



statistical research 









X 



stacisdes 

i 

+ .079 










status 

1 

+ .022 

X 









status quo 

1.2g 

-.592 

X 

X 

X X 

x 

X 


x 



status quo ante bellum 








X 




statute 

it 2g> s 

+ .188 

X 

X 





X 

z 


stay 

« « s • 









X 


steadiness of endeavor 

2s 











steal 

1 

+ .616 










steerage 

1 

-.122 










steenng committee 

)g 








s 


z 

stenlizadon 

1 

-.382 










steriing 

• • • « 










X 

steward 

1 

+ .127 










sdfle 

1 

+ .258 









B 

sdmuladon 

1, 2s, E 

+ .327 









« 

stockade 

1 

-.006 

X 








X 

ftock company 

1, 2e 

-.593 









a 

stock exchange 

1, 2e 

-.323 




X 

X 





stock holder 











X 

stocks 

S 








X 


z 

stoppage at source 

2e 











storage 

2e 










B 

store 

1 

+ .690 








z 


stories 

1 

+ .616 









• 

struts 

• • ■ • 










X 

stranger 

2s, E 










. 

strategic 

.... 










z 

strata 

1 

+ .126 

X 








X 

stradficadonn^ social 

2s, E 





X 

x 


X 


• 

straw vote 

• • • • 










z 

strict construedon 

1 

-.572 

X 

X 





X 



strife 

1 

+ .506 

X 









strike 

l,2e,S 

+ .506 

• 



X 

« 


* 


z 
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Wor^ 


itrike-breaker 

atringent 

ftronghold 

•tructural rcformi 

ftructurei 

ftru^e 

ituboornneat 

itudenta 

■tump ipeaker 

aub*aeacon 

lubdivifion 

subdue 

subjects 

subject states 

subjugation 

sublimation 

submarine 

submission 

subnormal 

subordinates 

subpoena 

subscription 

subservience 

subsidiary coin 

subsidiary 

subsidies 

substitution 

sub-treasury 

suburb 

sub-vaual 

success 

succession 

succor 

sue 

suffrage 

suffragette 

suggestion, soaal 

suggestibility 

suit 

suitor 

sulphurous 

Bulun 

suflunation 

summons 


3 

I 


S 


Wesley 

Weriey Tests, College 

Editions No. • — 


I « ^ > I 


1 1 


1.2e 

1 

1 

1 


-I- .062 
-.301 
+ .327 


+ .68:9 
+ .022 
-.090 
+ .206 
+ .443 


I, 2s. E 
E 

j... 

1 

1 

2g 

1 

I. 2s 
2e 
1 

1. 2g, 2e 
1 
1 
1 
1 

U 2s, S 
1 
1 
\ 

1, 2g, S 
1 

2s, E 

£ 

l.S 

I 

.... 

E 

2g 


1, 2s, E + .495 X 


-.522 X 
+ .083 X 


+ .269 
+ .020 
+ .211 

+ ‘.437 
-.231 

‘-.W 
-.363 
+ .363 
-.523 
+ .311 
-.754 
+ .553 
+ .385 
-.175 
+ .442 
+ 211 
+ .132 


+ .379 X 
+ .327 . 

-;647 X 


X 

X 

u 


X 

X 
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Word 

I 

c 

wC 

‘C 

e 


Wesley Tests, 
Editions No. 

Wesley 

College 

CO 

M 

G 

C 

t 

o 

s 

4 

s 


« ^ K > 

1 3 S 

D a. u: a. 

■ummoneef 






X 

luperficialitv 

1 

-.301 





lumptuary law 

Sunaay lawi 






X 

X . . . 

luperintendent 

1. 2g. S 

+ .437 

X 



X 

lupcriority 

1 

+ .495 

X 



. jr.h.s. . 

lupernatural 

1 

-.005 




. 

luper-orf^anic 

E 

+ jii 





tuperitition 

1 




X 

■uper-tax 

1 

-.977 

X 




luperviiion 

1 

2g. S 

+ .316 

X 




aupervisor 


X 




lupplant 

I 

+ .216 





auppliei 

1 

+ .558 

X 



X 

lupply and demand 

1 

-.609 





aupport 

S 

+ .632 

X 



X 

lapportcr 

1 

1, 2s, E 

X 

X 


X 

tupprestion 

+ .458 

X 



X 

tupremacy 

1 

+ .458 

X 



X 

Supreme Court 

1. 2g. S 

+ .316 

X 



2 

•urety 

1 

-.486 





turpaai 

1 

+ .400 





aurplice and ring 

1 

-1.139 





aurplus 

1, 2e 

+ .326 




X . X X 

aurplui, accumulated 






X . . . 

•urplut carningt 






X . . . 

aurplus value 

i 

-.987 





surplus wealth 






X . . . 

surrender 

i 

+ .690 

X 



2 

surrogate 

2g 







surroundings 

S 






sur-tax 

1, 2e 

-1.192 

X 



X 

surveyor 

2g 






survival 

1, 2s, E 

+ .316 





survival of the fittest 

1, 2s. £ 

+ .316 




X 

suspects 






X 

suspend 

i 

+ .569 

X 

X 



suspension 


-.929 




X X 

suspensive veto 

i 

X 




suspicion ^ 

1 

2s 

+ .574 




. . 5 , 

sustaining group 



* 



suxerain 

1 

-.993 

X 



X 

swashbuckling 

• . . . 





X 

sway 

1 

+ .400 

X 

■ 

. 

. 
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Wovd 


sweat shop 

swindle 

symbol 

sympathetic stake 

syndicalism 

syndicate 

synew 

synod 

system 

synthesis, social 

uble (v.) 

taboo 

tactics 

taille 

ules 

talesman 

talisman 

tame 

tannble investment 

tarilF 

unlT commission 
tariir dudes 
tariflF for revenue 
tariff preferential 
tariff, protective 
tariff rates 
unff schedule 
tariff treaty 
tariff union 
tax 

taxable taxpayer 
uxadon 
tax*dodger 
azfree minimum 
uxpayer 
uz rates 
Taylor System 
teacher 
teamwork 
technicality 
technical school 
technique, social 
telcsis 

telegraph 



s 

Wesley 

I 


Wesley Tests, College 

> • 

^ s 

Edmons No. - — * — ■ 

c 




S X 


(/3 

M 

£ ^ 

0 1 

1 i 3 1 


S i - 


1, 2e, S 

- 382 

X s 

1 

+ 448 

X 

1, 2s. E 

+ 137 X 


1 

- 493 


l,2e 

- 1 024 X 


1 

-.571 

X 

E 


. . • 

1 

- 9R7 

X 

1 , s 

+ 437 X 

. 

E 


. . • • 

1 . s 

+ 301 X 

X X 

1 

- 465 

X 

1 

- 269 X 

X 

1 

- 1 398 X 

a 

l.S 

- 958 

z . . I 

1 

- 861 X 

X . . ■ 

1 

+ 632 

X 

1, 2g, S 

2g 

+ 327 X 

X X . z 

1 

+ 194 

* 

1 

M 

C^ 

1 

• 

2e 

2e, 2g 



2g 



2g 


X 

1. 2c, 2g 
2g, S 

- 703 

z 

S 

1, 2g, S 

+ SS8 X 


1 

+ 241 X 


2e 


c 

1. 2g 

+ 747 X 

^ • 

X 

2e 


. 

1 

•f 747 


i,s 

+ .747 

■ « X s 

1 

+ 074 


1 

+ 253 . 

. 

E 

. 

- 

2s, E 



X .... 

. . . s 


[ 599 ] 


TESTS AMD MBASUMEMENTS 


Word 

List Symbol 

s 

-5 

s 

> 

e 

0 

1 
s 

Wesley List 

Wesley Testft 
Editions No. 


Wesley 

College 

a 

ft. -ft 
ft 9 

■V S 
^ a. 

1 

H 

1154 

III 

> 

telephone 


. . » 


. «. » » 




. ZX 

temperance movement 

1 

+ .385 


X 

X 




temperament 

2s 









templet 

I 

+ .390 







temporal 

1 

-.371 







temporal power 

1 

-.656 

X 






temptation 

1 

+ .558 






6 

tenant 

1 

+ .448 





X 


tenant farmer 

.... 







X 

tenant-in-chief 

1 

-.986 







tendency 

1. 2s 

+ .390 






X 

tender 








X 

tenement 

i.s 

+ .206 







tention 

E 








tenure 

1 

-.444 

X 





» » 

tenure of employment 

« . • • 






X 


tenure of office 

2g 


X 





s • 

term (office) 

1, 2S. S 

+ .616 






X X 

termination 

1 

+ .269 







terminal 

1 

+ .269 







terms (treaty) 

1 

+ .253 

X 





X X 

territorial expaniion 

I.S 

+ .069 

X 






territorial integrity 

1 

-.862 

X 

X X X X 

X 




territory 

1 2g.S 

+ .443 

X 




X 

. zz 

terror 

1 

+ .622 

X 

X . . . 




4 

terroriim 

1 

-.236 

X 






teat 

1 

+ .558 

X 






testator 








X 

teat oath 

i 

-.745 

X 






testify 

.... 







X 

testimony 

2g 


X 





z 

textile 








X 

theft 

i 

+ .616 







theology 

1 

-.408 

, 





x 

theocracy 

1.2g 

-.934 

X 

X X X X 





theory 

1 

+ .132 

X 






therapeutici 

.... 






X 


they-mup 
tiura desrw 

2i, E 

1 

-.396 






k • 

• • 

ditrd estate 

1 

-.951 

X 





« « 

third party ^ 

1.2s 

-.260 

X 





• 

threaten 

1 

+ .689 

X 





5 . 

riirift 

1.S 

+ .521 






z 

difone 

1 

+ .579 

X 





X 
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Word 


1 

I 

M3 


8 

g 

t 

o 

s 


Wedey 

Wedejr Tem, Collfp 

5 EditMMii No. • — • — 




thwart 

1 

+ 279 

t • # 

, , 

acker 

acket 

2g, S 



X 

X 

tillage 

time and motion studiei 
tithe 

1 

- 582 

X 

X 

tide 

i.s 

+ 132 


X X 

tide deeds 

1 

- 583 

• • 


tocsin 

1 

-1 003 . 


s 

toga 

1 

- 703 a 


X 

toleration 

1. 2s 

-1 387 


X 

coll 

I.S 

+ 063 * 


X 

tomahawk 

tomb 

1 

+ 579 

> • 

X 

tonnxge 

tooli 

J.28 

+ 006 

• s ■ 

a 

X 


torch bearer 1 — 

tonei 1 ~ 

torment 1 + 579 

torpedo 1 “ 374 

tort 1* 28 ~ ^ ^ 

torture 1 + 264 

totem 

totcmiim 2( 

tournament 1 

town 2g, S 

town meeting 1. 2gi S 

townihip Ii 2g> S 

tract 1 

trade 1» S 

trade agreement 1 

trade area 

trade association 1 

trade guilds 1 

trade, interstate 2e 

trade, intrasute 2e 

trademark 2g 

trade monopolies 1 

trade regulation 1 

trade route 1 

trade school 1 

tradesman 

trade union li b 

ttadhg post I 


i 

% X 


t . 


+ 127 

- 068 

- 023 
+ 243 
4- 632 

- 324 

- 382 

- 629 


- SM 

- n3 
+ 126 
+ 316 


X X 


+ 

+ 


258 

074 
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X X 
X 
t 


X X 
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Weslev 

College 


3 

a. 

c 




3 


a. 


tradition 

I. 2s, h 

+ 2h<i 

X 



7 

traffic 

1 

-f- 564 




X 

traifii aKsociatmn 

2c 

. 





training 

S 



X 



trait 

1 

+ 269 



X 


traitor 

1 

+ 448 

a 



X 

tranquillity 

1 

- 042 

z 




transaction 






X 

transcontinental 

1 

+ 379 




X 

transfer 

1 

+ 448 





transference of ideas 




X 



transformation 

1. 2s. E 






transit 

1 

+ 079 




X 

translation 

1 

+ 443 



X 


transmission of acquired 






characteristics 




X 



transmutation 

2s 






transport 






X 

transport afrency 




X 


z 

transportation 

1, S 

+ 443 


X 



transvaluation 




X 



transubstantiation 

1 

- 851 





travel 

1 

+ 616 




X 

traveler’s check 

1 

- 476 





treachery 

1 

+ 369 





treason 

1, 2g, S 

+ 258 

X 

XX X 


t 

treasure 

S 






treasurer 

2g,S 


, 




treasury 

1' ^ 

+ 369 

X 


X 

zz 

treasury notes 

2g, 2e 






treat (v.) 

1 

+ 336 

X 




treaty 

1.2g.S 

+ .258 

X 

X 

z 

z 

treaty of alliance 




X 



trea^ ports 

1. 2g 

- 708 


X 



treaty tone 



. 

X 



trench 






X 

trend 

1 

+ .090 

. 




trespass 

2g 



. 



tnai 

1, 2g 

+ .563 

X 



X 

trial by combat 

1 

- 765 

X 

. 


. 

trial by jury ' 

. , 



X 

X 

, 

tnbe 

1 

+ .501 

X 



X 

tribunal 

1 

-.333 

X 

. 


X 

tribune 

1 

-.756 

X X 

■ 4 • 




[ 602 ] 


Kelly 

PresKy 





APPENDIX II 


_ u 

Word E ^ 



1/5 

w 
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7 

4i 

tnbutary 

1 

- n? 


tnbute 

1 

+ 2t)9 


trickery 

1 

+ 443 


tricolor 

1 

- 818 

X 

tripartite 

triumph 

1 

+ S06 


triumvir 

triumvirate 

1 

- 729 

X 

troops 

1 

+ '74 

X 

tropics 

trouble 

1 

+ 747 

X 

trover 

truancy 

h 

1 

+ iOO 


truce 

l 

+ 327 

X 

Vtvmk. \iTies 

\,lt 

- m 


trust 

1. 2c, S 

-t- 04t. 


trust busting 

1 

- f)39 


trust company 

1, 2c 

- 724 


trustees 

1. 2c. 2 k, 

- 012 


trust fund 

2e 



tsar 

1 

- 021 

X 

tuition 

1 

+ SS8 


turbulence 

1 

+ 153 


turmoil 

1 

+ 269 

X 

tutor 

1 

+ 326 


twelve-hour day 

1 

+ 132 


two-party system 

1 

- 408 

X 

type 

1 

4- 379 


typical 

tyranny 

1. 2r, S 

+ 199 

X 

tyrant 

ukase 

1 

-1 140 

X 

ultimatum 

l,2g 

- 192 

X 

nltia-conseivative 


- 592 

X 

umpire 

1 

+ 432 


unadjustment 

unanimity 

unanimous 

E 

2k 

1 

+ 579 

X 

Unconstitutional 

1. 2g. S 

- Oil 

X 

underconsumption 

2e 



undercut 




understand 

i 

-h 558 


underworld 



underwriting 

2e 
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Kditioni No 
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C’ollegt 


3 

& 

e 


X 


X 

X 


5 


1 

X 

X 


X 

X 


X * . , X 

X jf h « «x 

* X 


X 



X 


X 

X 


X 

X 

X 



X 

s 

X 

% 
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underwriter .... x 


unearned increment 

1.2e 

-.982 







X 



unemployment 

1. 2e,S 

+ .448 





X 


X 


X 

unemployment, cyclical It 











unemployment iniur- 












ance 

1. 2e 

-.519 






x 




unemployment, teatonal 2e 











unfair liiu 

1 

-.719 










unfit 

1 

+ .517 










unification 

1 

+ .295 

X 









uniformity 

1 

+ .269 

X 









union 

1. 2g, S 

+ .443 

X 







4 

z 

unionize 

1 

-.047 










unique 

1 

+ .153 







X 


x 

unit 

1 

+ .269 

X 









unit rule 

2g 

1 1 1 





• * 



X 


unite 

1 

+ .437 





» * 





United Preia 

S 





» » 





United Statei 

United Statei treasury 

S 








z 



unity 

universal 

.... 

+ .264 






X 



X 

univenal education 
universal military 

1 

+ .206 










service 

1 

+ .019 

X 









universal pattern 

E 











universal suffrage 

1 

+ .089 

X 









iiniverse of discourse 

E 

+ .501 










university 

1 









X 

uniutt 

uxuawful 

1 

+ .564 

x 

X 

X 


X 






X 

X 








unofficial 

1 

+ .506 









X 

unorganized 

unortnodoz 

1 

+ .390 

X 






X 



unorganized labor 

i 

+ .132 










unprofiuble 

. , . t 










X 

unrepresentative 

s 









♦ 


unrest 

I, 2t 

+ .322 

X 









untcrupulousness 

1 

+.079 










unskilled labo^ 

1 

+ .390 





X X 





unsocial 

1 

-.138 









* 

unsound nuuiey 

m m • * 







X 



» 

untaxed wealth 

• • t • 








X 


a 

untrustworthy 









X 


• 


[ 604 ] 


Pub. 

Kelty 

Prewey 




APPENDIX 11 



jC 



Wesley 
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E 

% 

r 


Wesley Tests, 

College 


9 

Edmons No 

■ -■ 

m 

m 

§ 

c 

0 

s 

1 

I 

II 

1I>4 

ill 

IV 

ji 

1 - 
9 

Z) 0. 


unveiling 

1 

+ 190 

* • 


e 

upheaval 

1 

+ 327 a 



• 

upholder 

I 

+ 190 * 



e 

upper class 

1 

+ 257 



s 

upper house 

1 

+ 178 * 



• 

uprising 

1 

+ 448 * 



X 

upstart 

1 

+ .211 . 




urban 



. 


t 

urban centers 

1 . s 

+ .003 '. 

, . 



urbanization 

2s 


X 



urban population 

1 

+ 003 




usages 

1 

+ 148 

. 



usurpation 

1 

- 297 * 

* 



usuiy 



X 



urilitanan 

1 

- 814 

. 



utilities 

i.s 

- 454 . 

. 


X 

utility 

2e 


X X . X 



utilization 

1. 2s. E 

- 047 

. 



Utopians 

2e 

X 

' 



vacancy 

1 

+ 264 


6 


vacation 

i.s 

+ 747 

. 


X 

vagrant 

1 . s 

+ .148 * 

. 


X 

valid 

I.S 

+ 143 * . 

X 


X 

valor 

1 

+ 485 X 


6 

. 

valuation 

],2e, 2s,E,S + 264 X 

X 


X 


value 

vandalism 

vanguard 

vanquish 

variability 

variation 

variety 

vassal 

vassalage 

vault 

velocity of circulation 

venal 

veneer 

Venera rion 

vengeance 

vemte 

ventures 

veracity 

verdict 


- 445 

- 134 
+ 327 


+ 500 . 

- 454 * 

+ 327 ' 


- 107 . . . 

+ 148 

+ .442 - . . 

+ .269 ’. . 

- 054 

+ .269 ax* 
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e 

>v 

ft# 


S 

0 

s 

C 

1 

11 

III 

IV 

Unp 

Pub 


u 




veriBcation 



vernacular 

1 

-.507 

Versailles Treaty 

S 


verticle combination 

1 

-i.313 

vessel 



vested interests 

i.s 

-.950 

veteran 

1 

-I-.443 

veto 

1.2g 

-1-.49S 

veto item 



vexation 

i 

+ .343 

vice 

I 

+ .385 

vice and crime ring 



vice-president 

l,2g 

+ .437 

viceroy 

1 

-.634 

vice-royalty 

. . 


victims 

1 

+ .495 

victory 

1 

+ .632 

villa 

1 

-.260 

village 

1, 2g, S 

+ .422 

villain 

I 

-.259 

villains 

1 

-.270 

vindictiveness 

1 

-.281 

violation 

1 

+ .258 

violence 

1 

+ .553 

virtue 

1 

-.385 

visa 

1, 2g 

-.708 

vise 

S 


viscount 

1 

-.861 

visitation 

1 

-.044 

visiting nurse 

1 

+ .211 

vital statistics 

1, 2g 

-.508 

viva voce 

2g 


vizier 

1 

-1.053 

vocation 



vocational guidance 

i.s 

-.387 

void 

1. 2g 

-.216 

volition 

2s 


volley 



Volitoad Act 

S 


voluntary arbitration 

1 

-.369 

volunteer 

1, 2g 

+ .443 

volunteer army 

1 

+ .322 

volunteer clubs 



vote 

1. 2g,S 

+ .506 


X 

X 


X 

X 


X 

X 


X 

X 


X 

X 

z 

z 


zz 

X 

y 

X 
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CoUeie 


^ J -i 


voting machine 2g 

voting unit 

vow 1 

voyage 

vulnerability 1 

wage 1 

wage earner 1 

wa(K *cale 1 

walk-out 1 

want* 1, i 

war 1. 

war cloud 1 

war debt It 

ward off 1 

ward It 

warden It 

warehouce 1 

warehouse receipt 2e 

warfare 

warning 1 

war power 2g 

warship 

warrant It 

warrior 

Washington Conference S 
waste It 

wasteful consumption 2e 
wastes of competition 2e 
watcher 2g 

watchful waiting 1 

watered stock 1 

waterpower U 

waterpower site* 2^ 

waterway U 

Ways and Means 
Committee 

wealth If 

weapons 1 

weather bureau li 

we-feeiing ^ 

we-^up E 

welfare *i 

welfare worker 1 

well-to-do ' 


+ .327 


1 —.3m 

1 +.5M 

1 +.44S 

1 -.100 
1 -. 2 % 
l,2stE.S + .632 
1. 2g, S + .616 


1 , 28 

1 

1. 2g, S 

lf2g 

1 

2e 


+ .132 
+ .S0f. 
-.312 
+ .269 
+ .132 
+ .327 


1 +.569 X 

2g 

l, 2g. S + .137 X 


+ .437 


X 

-.713 X 
-.828 . 
-.099 . 

+ !i43 ! 


+ .374 
+ .632 
+ .390 


+ .327 
+ .074 
+ .390 


X 

6 . 

XX 
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1 
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Weriey Tests, 
Editions No. 

Wesley 

College 

1 

i 

s 

•8 


AI 

III 

Mil 

II 

& •§ 3 1 
;=> b ^ a. 

wergeid 

I 

-1.018 

X 



. . . . 

westernization 

1 

-.777 

X 



. ^ ^ M 

Westward Movement 

1 

-.339 

X 



P P P P 

wet 






X 

"what the traffic will 
bear" 

2e 






Whigs 

1 

-.966 

X 



s 

whip 

S 





X x 

whipping post 






X 

whiskey 






X . . . 

white man’s burden 

i 

1 

oo 

X 

X 

X . . . 

X 

White House 






X 

wholesale 

i 

+ .374 



• • p • 

. jr.h.s.x 

wholesome 

I 

+ .385 



• • t • 

. . 6 . 

wield 

1 

+ .201 

x 


• t » p 


wilderness 






. . 4 . 

wilt 

i.s 

+ .501 



p p • p 

X X 

will, social 

E 




p p p p 

S p • s 

wish 

E 




p p p ■ 


witan 

1 

-1.424 

X 



p » p 

witchcraft 

1 

-.175 




X 

withdraw 

1 

+ .501 

X 



X 

witness 

1. 2g. S 

+ .448 




X X z 

woman's suffrage 

1,S 

+ .263 

X 




work 

S 






working conditions 

1,2c 

+ .085 




X 

workman's compensa- 
* tion law 

i,s 

-.751 




x 

world 

S 






world court 

1 

-.306 

X 



X 

world despotism 

1 

-.851 

X 




world empire 

1 

-.403 

X 


» P B P 

p p p p 

world monarchy 

1 

-.582 

X 

X 

P » • • 

B S P p 

world peace 

1 

-.032 

X 


P P P • 

. . 6 . 

world power 

h2g 

-.259 

X 

X 


X 

worship 

1 

+ .385 




z 

worth 

1 

+ .443 





wounded 






• X 

wrangle 

1 

+.206 





wreck \ 

• ■ ■ ■ 





. . 4 . 

ssrest 

1 

+ .211 

X 


• P « B 

4 • * 4 

writ 

1.2g,S 

-.302 

X 


« • B * 

. . . X 

writings 

I 

+ .385 



4 • P P 

• • • • 

written constitution 

1 

.000 

X 



s 
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m 

n 
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t* 
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e 

s 

o 

s 

>s 

-a 

41 

it 

I 

II 
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III 

IV 

VJnpub. 
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I 


written law 

1 

- 148 X X 


wront doing 

1 

+ 327 

4 

yellow dog contract 



W 

"yellow" newspapers 

1 

- 402 


yeomanry 

1 

- 919 . . 

z 

yield 

1 

+ ^79 

. 

yoke 

1 

+ .326 X 

y 

Young Plan 

2g 




xeal 

1 

-.384 . . 


zealot 




Zeppelin 

zone 

i.s 

+ 327 X . 

X . . X jr.h.s. X 

zoomorphtsm 

2s 

. . 

. . 
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man, H., Wood, B. D. High schools and colleges. World 
Book Co., New York, 1928. 

American Council Euro^an History Test, Carman, H. J., Lan^am, 
W. D., Wood, B. D. High schools and colleges. World Book 
Co., New York, 1928. 

American History. Harlow' Publishing Co., Oklahoma City, Okla. 
High school. 

Anal^ical Scales of Attainment in American History. Educational 
Test Bureau, Minneapolis, Minnesota. Grades 7 to 8. 

Ancient and Medieval History. Harlow Publishing Co., Oklahoma 
City, Okla. High school. 

Barnard, A. F., Test in Roman History. University High School, 
University of Chicago. 

Barr, A. S., Diagnostic Tests in American History. Public School 
Publishing Co., Bloomington, 111. 1920. 

Barr-Daggeit Tests in American History. Educational Test Bureau, 
Minneapolis, Minn. Senior high school. 

Barr, A. S., Gifford, C. W., American History Word Meaning Test. 
Public School Publishing Co., Bloomington, 111. 1931. 

_Bell, J. Carleton, and McCollum, D. F., “A study of the attain- 
ments of pupils in United States history,” Journal of Educa- 
tional Psychology, VIII (May, 1917) 2S7-27A. 

Bowman United States History Test. Public School Publishing Co., 
Bloomington, III. 

Brooks, Fowler D., and Basset, S. Janet, “The retention of Amer- 
ican history in the junior high school,” Journal of Educational 
Research, XVIII ((October, 1928) 194-202. 

Broome, E. C., and Boyer, P. A., “Report on survey test in history 
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Buckingham, R R., “A proposed index of efficiency in teaching 
United States history,” Journal of Educational Research, 1 
(March, 1920) 161-171. 
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Clark, M. G., lests of historical sense in fourth and fifth grade 
pupils,’ Historical Outlook, XIV (April, 1923) 147-150. 
Columbia Research Bureau American History Test, Carman, H. J., 
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Test VI. Fifth Years of Progress, 1864-1914. 

Test VII. Recent Events, Including the World War and Its 
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Fofdyce Test for History VII (American) High School, Muskogee, 
Oklahoma. 

Fordyce Test for History IV (Ancient) High School, Muskogee, 
Oklahoma. 

Fordyce Test for History V (Medieval) High School, Muskogee, 
Oklahoma. 

Fordyce Test for History VI (Modem) High School, Muskogee, 
Oklahoma. 
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look, XII (December, 1921) 324-326. 
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Harvard Comprehensive Exams. — History. 
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Junior American History Test, Worid Book Company, New YorL 
Junior high school. 

Kelley, T. L., Test of Historical Ability^ Teachers College Contribu* 
tions to Education, No. 71. High school grades. 

Kepner, Tyler, "An aspect of historical testing,” Historical Out- 
look, XV (December, 1924) 414-416. 

Kepner, P. T., “A survey of the test movement in history,” Jour- 
nal of Educational Research, VII (April, 1923) 309-325. 

Knowlton, D. C., "Improving the quality of instruction in history 
with the aid of the photoplay. The development of teaching 
procedures and teacning techniques,” Historical Outlook, XX 
(April, 1929) 167-179; ^ay, 1929) 229-230. 

Koch, Julia F., "An adaptation of the standardized test to medi- 
eval history,” Education, L (December, 1929) 232-242. 

Krey, A. C., "What does the new-type examination measure in 
histo^?” Historical Outlook, XIX (April, 1928) 159-162. 

Krey, A. C., Wesley, E. B., "Does the new type test measure re- 
sults of instruction in the social sciences? A report of an ex- 
periment,” Historical Outlook, XXIII (January, 1932) 7-21. 

Lindquist, E. F., and Anderson, H. R., “Objectives testing in 
world history,” Historical Outlook, XXI (March, 1930) 115- 
122 . 

Lindquist, A. E., "The form of the American History Examina- 
tion of the Cooperative Test Service,” Educational Record, 
XII (Oct., 1931) 459-475. 

Lindquist, E. F., and Anderson, H. R., "The improvement of ob- 
jective testing in history,” Second Yearbook of the National 
Council for th^e Social Studies, pp. 97-117, 1932. 

Lindquist, E. F., and Anderson, H. R., "Achievement Tests in the 
Social Studies,” Educational Record (April, 1933). 

.Lynch, I, R., Lynch American History Test Series. Educational 
Test Bureau, Minneapolis, 1926. Grade 8. 

Martin, D. H,, "A comparison of objective examinations in Amer- 
ican history,” American School Board Journal, LXXVI, no. 4 
(April, 1928) 49, 130, 132. 

Maryland State Department of Education, History Tests for 
Grades 4-7. 

McCollum, D. F., Test in American History. Journal of Educa- 
tional Psychology, Vol. VIII, 257-74. 

Metropolitan Achievement Tests: New York City Edition — History. 
World Bodk Co., Yonkers. Grades 4-8. 

Michell, E., Teaching Values in New Type History Tests. World 
Book Co., Yonkers. 1930. 
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Minnesota State Board Examinations in Ancient and Modem His- 
tory, May, 1928. 

Mod^n European History. Harlow Publishing Co., Oklahoma 
City, Okla. High school. 

Modern European History Test. Center for Psychological Service, 
Washington, D. C. High school or college. 

Modem School Achievement Tests — History and Civics. Bureau of 
Publications, Teachers College. Clradcs 3-9. 

Monroe, Walter S., “Historical information and judgment in 
pupils of elementary schools,” Journal of Educational Research, 
III (April, 1921) 303. 

Monroe, Walter S., “Written examinations and their improve- 
ment,” Historical Outlook, XIV (November, 1923) 306-M8. 

Moyer, F. E., “New types of history tests,” Historical Outlook, 
XIV (November, 1923) 323-327. 

Myers, Carry C., “Delayed recall in history,” Journal of Educa- 
tional Psychology, Vlll (May, 1917) 275-283. 

New Stanford Achievement Tests — History and Civics. World Book 
Co., Yonkers. Grades 4-9. 

Odell, C. W., “The Barr Diagnostic Tests in American History,” 
School and Society, XVI (October 28, 1922) 501-503. 

Pencil, Olivia C., “Research tests in United States history,” His- 
torical Outlook, XV (March, 1924) 128-143. 

Peters, Charles C., and Altman, John E., “A critical study of the 
content of standardized tests in American History,” Journal of 
Educational Research, XXIIl (February, 1931) 153-161. 

Philadelphia Public Schools, Division of Educational Research, 
“Report on a survey test in history of the United States,” 
Historical Outlook, XX (January, 1929) 17-23. 

Pratt, Orville C., “Spokane United States History Test,” Journal 
of Educational Research, III (February, 1921) 155-157. 

Pressey, L. C., Test of Special V ocabulary of History. Public School 
Publishing Company, Bloomington, ill. 

Pressey, L.C. , Richards, R.C., U nder standing of A merican History. Pub- 
lic School Publishing Company, Bloomington, 111. Grades 6^12. 

Pressey, S. L., “An examination for measurement in history, arith- 
metic and English,” Journal of Educational Research, III 
(May, 1921) 359-69. 

Public School Achievement Test-History. Public School Publishing 
Co., Bloomington, 111., 1928. Grades 5-8. 

Ralston, Sue W.>“The use of supplementary readings and fact tests 
in connection with a one year course in ancient and medieval 
history Historical Outlook, XV (December, 1924) 410414. 
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Rayner, American History Test. University of Illinois Bureau of 
Research. Elementary mdes. 

Renfrote Tests in European Background to American History. C. A. 
Gr^iy G>., Cincinnati. Grade 6. 

Ruch, G. M., General Information Tests in America^ History. 
Commonwealth investigation exam’s soc. sd. University of 
Iowa. 

Rugg. Earle U., “Character and value of standardized tests in his- 
tory,” School Reoiew, XXVII (December, 1919) 7S1~772\ or 
Historical Outlook, (June, 1920) 231-236. 

Sackett, Leroy W., “A scale in ancient history,” Journal of Edu- 
cational Psychology VIII (May, 1917) 284-293. 

Sackett, Leroy W., “A scale in United States history,” Journal of 
Educational Psychology, X (September, 1919) 34S-348. 

Shryock, Richard H., “New tests for old. An examination of the 
relationship of aims and methods of testing in the teaching of 
history,” Historical Outlook, XIV (November, 1923) 319-322. 

Shyer Test in World History. Palmer Co., 120 Boylston St., 
Boston. 

SpokaneH istory Test (O. C. Pratt, Superintendent). 

Starch, D., American History Tests. IJniversity Cooperative Co., 
Madison, Wis. 

Stormzand, M, J., “Tests in American history,” American History 
Teaching and Testing. Macmillan, New York, 1925. 50fF. 

Thornton, E. W., “The use of4nformational tests in American his- 
tory teaching,” Historical Outlook, XX (January, 1929) 12-16. 

Tyrrell American History Exercises, 120 Boylston St., Boston. 

United States History. Harlow Publishing Co., Oklahoma City, 
Okla. Grades 7-8. 

Unit Scales of Attainment — American History, UniUd States His- 
tory. Educational Test Bureau, Minneapolis. American, 
grades 3-6; United States, grades 7-8. 

Van Bibber, L. C., “Study of specific classroom difEculties in 
teaching history,” Second Yearbook, National Council for 
Social Studies, 1932. 

Vannest, C. G., Diagnostic Test in Modern European History. 
Indiana University Bookstore, Bloomington, Ind. High 
school grades. 

Van Wagenen, M. J., American History Scales. Bureau of Publica- 
tions, Teaphers College, 1919. Grades 5-6. 

Van Wa^«i,^. J., ‘The Van Wagenen reading scales in history, 
general sdence, and English literature,” Journal of Educa- 
tional Research, III (April, 1921) 314-316. 
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caver, Robert B., and 1 raler, Arthur t., ‘*Kssay examinations 
and objective tests in United States history in the junior high 
school, School Rn'iru, XXXIX iNovenihcr, 1931) ^9-6^. 

Wessel, Herman M., “A critical studv of the results of Barr’s Di- 
agnqutic Tests in American History." Ilisioricql Outlook. XVII 
(May, 1926) 230-31. 

Williams, L. W., "Barr’s Diagnostic li-Ms in American History," 
Journal of Educational Research. Ill ( January, 1921) 74-75. 

Wilson, H. C., and Waples, D., C lass room procedure test in Aijfory 
and other social studies. University of Chicago Press, 1928. 

Wilson, H. E., " 1 he continuity test In history teaching," School 
Review, XXXIV (November, 1926) 679-684. 

Witham, E. C., Com prehens he History Tats, J. 1.. Hammett Com- 
pany, Cambridge, Mass. Grades 7 and 8. 

Wood, B. D., The 1 . E. R. College Entrance Examinations tn An- 
cient History. 'I'eachers College, Bureau of Educational Re- 
search, 1922. 

Civics and Government 

% 

AbeJow, S. P., "A group test of civics ability,” Historical Outlook, 
3^11 (November, 1931) 345-349. 

Almack Tests in American Civics and Government. C. A. Gregory 
Co., Cincinnati. High School. 

American Civics and Government Test for High Schools. Public 
School Publishing Co., Bloomington, 111. 

American Council Civics and Government Test — Leigh, R. D„ Me- 
Goldrick, J. D., Odegard, P. H., Wood, B. D., World Book 
Company, 1929. High schools and colleges. 

American Government and Civics. Harlow rubUshing Co., Okla- 
homa City, Okla. 

Barr, A. S., "Measurements in civics," Historical Outlook, XV 
(January, 1926) 24-26. 

Bear Test on the>Vnited States Constitution. Public School Publish- 
ing Co., Bloofhington, III. 

Brown, A. W., and Woody, C., Brown-Woody Civics T'St. World 
Book Co., 1928. 

Burton, W. H., and V. N., Civics Test. World Book Company, 

1228 - . . . L 

Cannon, Annie E., "A plan for teaching a course in civics on the 
unit basis,” Historical Outlook, XVI (January, 1925) 28-35. 

Chassell-Upton Citizenship Scales. Teachers College Bureau of 
Publications, 1922. 

Civics. Harlow Publishing Co., Oklahoma City, Okla. 
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Dawson, Edgar, “The history inquiry . . . VII. Experimenting 
with a test,*' Historical Outlook^. XV (June, 1924) 260-268. 
Gibbons, A. N., Civics Final Test^ 

Background Test in Civics 

Harvard Comprehensive Exams. — Government. AH pubKshed by 
National Council for the Social Studies, March, 1928. 

Politics 

Hill, H. C., A Test in Civic Information. Public School Publishing 
Company, 1926.' Grades 6-12. 

Hill, H. (i., and Wilson, H. E., Test in Civic Action. Public School 
Publishing Company, Bloomington, 111., 1929. Grades 6-12. 
Iowa Academic Contest, 1930; Examination in American Govern- 
ment. 

i ohnstown Public Schools, Community Civics Test. Johnstownj Pa. 
linneapolis Public Schools Curriculum Department, Test in Un- 
derstanding of Ideas Used in Community Life Problems Course. 
Monroe, Walter S., “Written examinations and their improve- 
ment,” Historical Outlook^ XTV (November, 1925) 306-318.' 
New York State Civics Test. State Education Department, Al- 
bany, 1928. 

Rugg, H. O., Vocational Civics, Grade 7B-7A. Based on An Intro- 
duction to American Civilhation. Ginn Si Co., 1929. Test by 
Miss Tulley, Providence, R. I. 

Stack, H. J., “Standardized tests in community and economic 
civics,” j^iftortcd/ Outlook, XVIII (April, 1927) 166-172. 
Stephenson, O. W., and McGehre, W. R., “A Vocabulary common 
to citizenship and American history,” Journal of Educational 
Research, XaII (June, 1930) 55. 

U pton-Ckassell Scale for Measuring Habits of Goo</ Citizenship. 
Teachers College Bureau of Publications, 1922. 

Economics 

American Council Economics Test (H. Taylor, T^TN. BarroifW, B. D. 

Wood) World Book Company, 1929. 

Iowa Academic Contest, 1930: Objective Examination in Principles 
of Economics. 

Komhauser, A. W., “Results from the use of objective top and 
questionnaires in The Economic Order,” Journal of austfuss 
of the University of Chicago, I (October, 1928) 429-455; 11 
(laniMry, 1929) 82-111. „ o . , 

Sins, V. M., ^are Card for SocichEconomic Status. Public School 
Publishing Co., Bloomington, 111., 1927. 
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Anat'^ical Scal'f of Jitamm-nt m G'o^rapky, Educational Teat 
Bureau, Minneapolis. 

Barthelmess, H. M,, "GeoRraphv testing in Boston.’* Journal of 
Educatnna! R'srarch, II t November, 192l)i 701-712. 

Branom, M. E., Diagnostic Tests in Geography. McNiehc and 
McNight. Normal. III., 1926. Grades 5-8.' 

Buckingham, B. R., and 8tt*\'cnson, 1*. R., Geography TesU United 
States. Information — I’rohlems. Public Scho^il Publishing 
Co., Bloomington, 111. 

Buckingham, B. R.. Stevenson. P. R., Place Geography Tests. 
Public School Publi.shing Bloomington, III., 1922. 

Cavins, L. \ “An experiment with standardized tests in a state 
teachers examination, ’ Journal of Educational Research, XIV 
(October, 1926) 2(K>-12. 

Courtis, S. A., Standard Test in Geography. Octroit. Mich. Ele- 
mentary grades. 

Detroit Public Schools, Department of Research 

Geography Test / b S - riie world, oceans and continents 
Geography Test 2 b S - Location, states 
Geography T.^st 3 b N -Location, countries 
Geography Test •/ b S — Location, countries of Europe 
Geography Test 5 b 6* —Location, countries and cities of western 
hemisphere. 

Forney, E. B., Test in Map-Reading .Ibilitiec (»inn & C’o., 1920. 

Geography Test. Harlow Publishing Co„ Oklahoma City, Okla. 
Grades 7-8. 

Gregory, C. A., Spencer, P. L., Geography Tes.s. University of 
Cincinnati, 1923. Grades 6-8. 

Geography Tests. Plymouth Press, Chicago, 111. Elementary grades. 

Gregory-Haggerty Geography Tests. C. A. Gregory Co., Cincinnati. 
Grades 4-6. 

Hahn, H. H., Lackev, E. E., Geography Scale. Public School Pub- 
lishing Co., 1928. Elementary grades. 

Hartwell, Geography Test. Mentzer, Bush and Co., Chicago. 

Industrial Commercial Geography. Harlow Publishing Co., Okla- 
homa City, Okla, High school. 

Information-Problems Test in Geography. Public School Publishing 
Co., Bloomington, III. Elementary grades. 

Johnstown (Pa.) Public Schools, Geography Test. 
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Lackey, £. £., "A scale for measuring the ability of children in 
gco^raph^,” journal of Educational Psychology ^ II (October, 

Mathewson, Chester A., “A preliminary note on the use of the 
Hahn-Lackey Geography Scale,” Journal of Educational Psy- 
chology, IX (October, 1918) 467. 

Mathewson, Chester A., “Source results with the Hahn-Lackey 
Scale in Geography,” Journal of Educational Psychology, II 
(December, 1918) 581-87. 

Metropolitan Achievement Tests: New York City Edition — Geography. 
World Book Co., Yonkers. 

Modern School Achievement Tests — Geography. Bureau of Publica- 
tions, Teachers College. 

New Stanford Geography Test. World Book Co., Yonkers. Grades 
4-9. 

Olmstead, M. C., Diagnostic Geography Test. 

Junior High School Ciarkston, Washington. Elementary 
grades. 

Place Geography Tests. Public School Publishing Co., Blooming- 
ton, 111. 

Posey, C. J., Van Wagenen, M. J., Posey-Van Wagenen Geography 
Scales. Public School Publishing Company, Bloomington, 111., 
1922. Grades 5-8. 

Public School Achievement Test — Geography. J. S. Orleans, Public 
School Publishing Co., Bloomington, 111., 1928. Grades 6-8. 

Russell-11 arr Unit Tests in Geography. Bureau of Educational Mea- 
surements and Standards, Kansas State Teachers College, 
Emporia, Kansas. Grades 4-8. 

Stevenson, P. R., “A problem test in geography (a preliminary 

. statement),” Journal of Educational Research, V (April, 1922) 
350-353. 

Stevenson, P. R., “Minimum essentials in place geography,” 
Journal of Educational Research,W (November, 1922) 300-306 

Unit Scale of Attainment — Geography. Educational Test Bureau, 
Minneapolis. 

WiedAcld-Walther Geography Test. World Book Co., Yonkers. 
Grades 4-8. 

Wiiham Geography Test of Europe. Southern California School Book 
Dfeposjtory^ Hollywood, Calif. Grades 6-9. 

Witham Geo^aphyTest of the United States. Southern California 
School Book Depository, Hollywood, Calif. Grades 5-8. 
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^ H**"*"”^ Co.. Cambridge, 

fFitham T^ts ir H'orid Qfogra^ks^ Southern California School 
Book Depositors, Mollvwood. Calif 


The Soria/ Studies 
(.general field of) 

Allport, F. H , and G W. “Personaluv Traits,’' Journal of Ah' 
normal and Social Ps\chnlQ^\, 1921. 

Allport, W. and \ erncm, F’ f , .■/ S*iid\ of I'alues. Houghton 
Mifflin, 1931 

Alpha Group £xamina/ion-U S S, Medical Dept, Division of 
Psvchologs, 1918 

Assoaation Press~/?(/>/r kuouled^e Test, .SVnrj- of Character and 
PfTSonalit\ y^f/f Ntw ^ ork, ^47 Madison Avc 

Association ?vt^s~Charaiter Cnnith Tnis, CM\st\an IforldCxtizen- 
skip, 1926-27 

Association Press— 6Aarai/rr (,rmttk Tests, Croup Relahonshtps, 
1926-27 

Association ?Tt<ii~Character Grouth Tests, Immediate Life SituO' 
ttons, 1926-27. 

Association Press — Character Grotith Tefts—Reltetous Concepts. 
192f27 ^ 

Association Press — General questions for Boys 1927-28. 

Association Press — Health Knoziledge Test 

Association Press — Interest Analysts 

Association Press— roc/’ Test of Biblical Information. 

Association Press — Lehman, H C, Lehman Plav Quiz. 

Association Press — Watson, G B , Some Methods of Program Ap- 
praisal. 1926. 

Association Press — Watson, G B , Rating Scales. 1928. 

Association Press — Watson, G. B., Experiments with Religious Edu- 
cation Tests. 1926. 

Association Press — Watson, G B , Some Accomplishments in Sum- 
mer Camps. 1926 

Association Press— of Religious Thinking, Forms A, E. 

Atlanta Public Schools, Department of Testing and Guidance, 
Atlanta Social Science Examination. 

Anonymous, “Finding list of hopeful undertakings," Historical Out- 
look, XIII (December. 1922) 321-326. 

Baker, Harry J., Telling fPhai I Do Test Public School Publishing 
^ , Bloomington, III-, 1930. 
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Beme, Esther Van Qeave, An Experimental Investigation of Social 
Behavior Patterns in Young Children. 1928. Unpub. Ph.D. 
Thesis. University of Iowa. 

Berne, Esther Van Cleave, An / nvestigation of the W ants of Children. 

Iowa Child Welfare Research Station, 1929. 

Bibliography of Objective Tests in the Social Sciences. University of 
the State of New York. 

Bowman, K. M., Mental Disease Research Questionnaire. (Revised 
June IS, 1929). Boston Psychopathic Hospital. 

Bureau of Personnel Research, Downey, J. E., Test II. 1920. 
Bureau of Personnel Research, Test XI. 1920. 

Burton Social Science Civics Tests, C. A. Gregory Co., Cincinnati. 
Grades 7-12. 

Character Education Institution, The Characteristics of Human 
Beings. National Capital Press, Washington, D. C. 

Chassell, J. O., Experience f'ariables. University of Rochester 
Medical School, 1928. 

Columbia Course in Contemporary Civilization, Test in Midyear, 
1925. 

Cornell, E. L., and Coxe, W. W., Orleans, J. S., Rating Scale for 
School Habits. World Book Company, Yonkers, 1927. 
Cowdery, Karl M., Interest Report Blank. Revision of C. I. T. 

Bureau of Personnel Research Interest Analysis Blank. 
Denver Public Schools, Curriculum Tests in Social Science, Grades 
7B, 7A, 8B, 8A, 9B, 9A. 

Detroit Public Schools, Character Education in Detroit, Report of 
Committee on Character Education. 1927. 

Detroit Public Schools, Department of Research 

Social Science Test VI, form A — Europe, by M. E. Irwin and 
L. Spencer 

Social Science Test VII, form A — U. S., by M. E. Irwin and 
L. Spencer 

Social Science Test VIII, form A — Grade 5 A by M. E. Irwin 
and L. Spencer 

Social Science Test X, Part I — Foods. Grade 3b, by S authors. 

Part II — Shelter and Fuel. Grade 3B, 
5 authors. 

Part III— Clothing. Grade 3B, 5 
authors. 

Social Science Vocabulary. 

Detroit Teachets College, How Children Choose Friends. 1929. 
Dimock, H, S., Hendry, C. E., Camping and Character. Association 
Press, 1929, New York. 
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